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advancement of aeronautics and space science. The 
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this important role. 

The NASA STI program operates under the auspices 
of the Agency Chief Information Officer. It collects, 
organizes, provides for archiving, and disseminates 
NASA’s STI. The NASA STI program provides access 
to the NTRS Registered and its public interface, the 
NASA Technical Reports Server, thus providing one 
of the largest collections of aeronautical and space sci- 
ence STI in the world. Results are published in both 
non-NASA channels and by NASA in the NASA STI 
Report Series, which includes the following report 
types: 

• TECHNICAL PUBLICATION. Reports of 
completed research or a major significant phase 
of research that present the results of NASA 
Programs and include extensive data or theo- 
retical analysis. Includes compilations of signifi- 
cant scientific and technical data and information 
deemed to be of continuing reference value. NASA 
counter-part of peer-reviewed formal professional 
papers but has less stringent limitations on manu- 
script length and extent of graphic presentations. 

• TECHNICAL MEMORANDUM. Scientific and 
technical findings that are preliminary or of 
specialized interest, e.g., quick release reports, 
working papers, and bibliographies that contain 
minimal annotation. Does not contain extensive 
analysis. 

• CONTRACTOR REPORT. Scientific and tech- 
nical findings by NASA-sponsored contractors and 
grantees. 


• CONFERENCE PUBLICATION. Collected 
papers from scientific and technical conferences, 
symposia, seminars, or other meetings sponsored 
or co-sponsored by NASA. 

• SPECIAL PUBLICATION. Scientific, technical, 
or historical information from NASA programs, 
projects, and missions, often concerned with 
subjects having substantial public interest. 

• TECHNICAL TRANSLATION. English-language 
translations of foreign scientific and technical 
material pertinent to NASA’s mission. 

Specialized services also include organizing and 
publishing research results, distributing special- 
ized research announcements and feeds, providing 
information desk and personal search support, and 
enabling data exchange services. 

For more information about the NASA STI program, 
see the following: 

• Access the NASA STI program home page at 
http://www.sti.nasa.gov 

• E-mail your question to help@sti.nasa. gov 

• Phone the NASA STI Information Desk at 
757-864-9658 

• Write to: 

NASA STI Information Desk 
Mail Stop 148 

NASA Langley Research Center 
Hampton, VA 23681-2199 
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Preface 


T his handbook provides guidance based on best practices for the plan- 
ning, preparation, review, reporting, and closeout of Standing Review 
Board (SRB) activities. Revision A updates the baseline version of the 
SRB Handbook published in 2009 to incorporate process improvements 
and changes in NASA’s Life-Cycle Review requirements and review guid- 
ance documented in NPR 7120. 5E, NASA Space Flight Program and Project 
Management Requirements. This handbook is consistent with the NASA 
Space Flight Program and Project Management Handbook (PM Handbook) 
issued by the Office of the Chief Engineer concurrently with this version 
of the NASA Standing Review Board Handbook as a companion document 
to NPR 7120.5. The SRB content in both handbooks is complementary; 
however, the PM Handbook contains a summary of SRB processes from the 
standpoint of the program or project manager, while this handbook provides 
more details of SRB processes and best practices for conducting independent 
assessments for SRBs and other participants in the LCR process. This hand- 
book also provides review guidance and best practices to most effectively 
administer and satisfy the program and project review requirements estab- 
lished in NPR 7123. IB, NASA Systems Engineering Processes and Require- 
ments. 

NASA implements space flight programs and projects of various sizes and 
complexity and requires them all to undergo LCRs. The overall program or 
project life cycle includes two categories of reviews: 

• The internal reviews conducted by the program or project as defined and 
maintained in the program or project plan. 

• The independent reviews conducted by the SRB as defined in the terms of 
reference. 

NPR 7120.5 introduces the concept of SRBs performing independent assess- 
ments of space flight programs and projects as part of the LCRs to help 
increase the likelihood of success. NPR 7120.5 requires the program or 
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The SRB is the board responsible for 
conducting independent reviews (life 
cycle and special) of a program or 
project and for providing objective, 
expert judgments to the Convening 
Authorities. 


project and an independent SRB to conduct most, but not all, of the LCRs. 
The body of this document addresses the designated SRB reviews. 

As a companion to NPR 7120.5, this handbook focuses solely on space flight 
programs and projects. Programs and projects governed by other NASA 
procedural requirements, such as NPR 7120.7, NASA Information Technology 
and Institutional Infrastructure Program and Project Management Require- 
ments and NPR 7120.8, NASA Research and Technology Program and Project 
Management Requirements, that need independent review can use this 
handbook for reference. For projects that do not follow the typical NASA life 
cycle, such as those involving commercialization, reimbursable agreements, 
and foreign partnerships, implementation of this handbook’s guidance can 
be adjusted to match the specific program’s or project’s review needs. 

NPR 7120.5 assigns responsibility for the independent reviews performed 
by SRBs to the Independent Program Assessment Office (IPAO) and the 
Centers. The IPAO is responsible for independent reviews of all programs, 
all Category 1 projects, and Category 2 projects with a life-cycle cost greater 
than or equal to $250 million. These reviews are Agency-level reviews. Host 
Centers are responsible for independent reviews of Category 3 projects 
and Category 2 projects with a life- cycle cost less than $250 million; these 
reviews are Center-level reviews. The Decision Authority may alter these 
criteria. 


Centers and other organizations 
using an SRB or equivalent 
independent review board should 
use this handbook as guidance 
and adjust the Agency-level 
specific content to the Center's 
review processes, practices, and 
organizational structure. 


Centers and other organizations using an SRB or equivalent independent 
review board should use this handbook as guidance and adjust the Agency- 
level specific content to the Center’s review processes, practices, and organi- 
zational structure. 

The SRB Handbook consists of five chapters: 

• Chapter 1 provides the context for the process of independent LCRs and 
identifies major principles of the SRB process derived from best prac- 
tices. It defines the governance of SRBs throughout the life cycle of the 
program or project. 

• Chapter 2 defines the highest-level principles that govern SRBs. It 
includes a discussion of the SRB review criteria and a list of SRB stake- 
holders, and defines SRB participation in reviews for the different types 
of programs. 

• Chapter 3 establishes the guidelines for the formation of SRBs for the 
different NASA programs and projects. It describes the three possible 
SRB structures and outlines the means by which SRB members and 
consultants-to-the-board are qualified to serve. 
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• Chapter 4 provides a description of the LCR processes from beginning to 
end. 

• Chapter 5 discusses the products and responsibilities of the SRB. It 
provides examples of program and project assessment guidance and 
details the six SRB assessment criteria. 

The appendices include examples and templates for the products identified, 
as well as reference material for SRBs that supplement the core chapters. 

Note that this handbook uses the word “independence” in broad terms, and 
it encompasses the term “independent” that is used extensively in NASA 
policy and requirements documents. 
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Introduction 


A s a key element in the National Aeronautics and Space Adminis- 
tration’s (NASA’s) strategic framework for managing space flight 
programs, Standing Review Boards (SRBs) help ensure appropriate program 
and project management oversight in order to increase the likelihood of 
mission success. This chapter explains the purpose and applicability of this 
SRB Handbook, notes SRB governance, and lists key guidelines considered 
major principles underlying SRB processes and products. 


1.1 Purpose of This Handbook 


This SRB Handbook provides review guidance for the program and project 
communities and for the SRBs regarding the expectations, processes, prod- 
ucts, timelines, and working interfaces with review organizations, Centers, 
Mission Directorates, Mission Support Organizations, and Management 
Councils. It provides guidelines for membership selection, review imple- 
mentation, review products, and reporting of results. 

The SRB Handbook guidance may be tailored, with the Convening Authori- 
ties’ approval, to meet the needs of the Agency, Mission Directorates, 
Centers, and the programs and projects being reviewed. The final review 
agreement of the SRB, program and project, and Convening Authorities for 
program and project Life-Cycle Reviews (LCRs) is documented in the terms 
of reference, as described in Section 4.1 and Appendix H . 


Independent reviews conducted 
by SRBs are key elements in the 
Agency's oversight of programs and 
projects. 


1.2 SRB Governance and Convening Authorities 


NPR 7120.5, NASA Space Flight Program and Project Management Require- 
ments, the governing document for LCR processes and products, establishes 
the requirement for an SRB to perform an independent assessment of a 
space flight program or project at specific LCRs. NPR 7120.5 governance of 
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1.3 ApplicabilityoftheHandbook 


NPR 7120.5 requires SRBs to 
conduct specific life-cycle reviews. 
The SRBs are convened by NASA 
senior management officials, called 
Convening Authorities. 


the SRB is derived from and consistent with NPD 1000.0, NASA Governance 
and Strategic Management Handbook. 

The traceability matrix of the SRB-specific requirements in NPR 7120.5 is 
located in Appendix G . 

SRBs are convened by the management officials, called Convening Authori- 
ties, identified in Table 2-2 in NPR 7120.5. This table defines the partici- 
pation and role of each Convening Authority in establishing the SRBs 
for programs and for each of the three categories of projects. Specifically, 
the table defines who is responsible for convening each type of review, 
approving the terms of reference, approving the SRB chair, and approving 
SRB membership. These individuals are the management officials who 
receive the briefings and documented results of the SRB. 

In addition to the standard LCRs, the Convening Authorities can authorize 
an SRB to conduct special reviews as needed. 


1.3 ApplicabilityoftheHandbook 

This handbook focuses on Independent Program Assessment Office (IPAO) 
implementation of SRBs for independent reviews. Centers and any other 
organization using an SRB or equivalent independent review board should 
use this document as guidance and adjust the IPAO-specific content to its 
own review processes, practices, and organizational structure. 

The program and project life-cycle figures in NPR 7120.5 identify the typical 
LCRs that require independent SRB review. 


1.4 Major Principles 

This handbook presents guidelines derived from best practices for SRB 
processes and products. Some of these guidelines are worth noting as “major 
principles.” Additional principles are noted throughout this handbook. 

1.4.1 General Guidance 

NPD 1000.5, Policy for NASA Acquisition, NPR 7120.5, and NPR 7123.1, 
NASA Systems Engineering Processes and Requirements define the LCR 
requirements; the PM Handbook provides guidance on requirements’ imple- 
mentation; this handbook describes how to implement the SRB process and 
assess the LCR requirements. 
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1.4 Major Principles 


1.4.2 Purpose of the SRB 

1. The LCR Convening Authorities are the explicit customers of the SRB; 
the program or project under review is the implicit customer of the SRB. 
The schedule of work performed by the SRB should support the needs of 
those customers. 

2. SRBs have an advisory role. The SRB conducts the LCRs and can 
provide recommendations, but the SRB members and consultants-to- 
the-board do not impose requirements on, make decisions for, or direct 
the program or project. 

3. The SRB is intended to promote Agency mission success. 

1.4.3 SRB Membership 

1. The Programmatic Assessment Group (PAG) Analyst and the Review 
Manager are SRB members. 

2. For Agency-level reviews, the Review Manager and programmatic 
analysts are assigned by the IPAO. For Center-level reviews, the host 
Center Director is responsible for assigning these personnel. 

3. The SRB is intended to have the same core membership through its 
engagement with the program or project, although its membership 
could be adjusted with specialized reviewers. For Center-level reviews, 
Center- specific processes should be used for Center SRB membership 
approval. 

4. The SRB remains intact for the duration of the programs. For projects, 
the SRB completes its work after the Operational Readiness Review 
(ORR). 

1.4.4 Roles and Responsibilities of the SRB 

1. The SRB chair and the Review Manager manage the content and 
schedule of work performed by the SRB. 

2. The SRB chair and the Review Manager coordinate the SRB’s activi- 
ties with the program or project to minimize the resource and schedule 
impact while fulfilling the LCR and SRB requirements, e.g., SRB 
members or consultants-to-the-board may attend program or project 
reviews rather than the SRB chair requesting special sessions. 

3. For each Agency-level review, the IPAO and the Mission Directorate 
collaborate to develop a budget addressing civil servant and contractor 
travel, labor, and procurement costs. 

4. The SRB briefs the program or project on its preliminary findings at the 
conclusion of the site review. 
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1.4 Major Principles 

1.4.5 SRB Independence and Integrity 

1. Apart from the Organizational Conflicts of Interest (OCI) and the 
Personal Conflicts of Interest (PCI) review and clearance process 
discussed in Chapter 3 and Appendices C and D, this handbook is 
strictly advisory; it is not a requirements document. It provides field- 
proven best practice guidance. 

2. The SRB functions independently of the program or project. SRB 
members are selected from outside the program or project management 
chain and are free of any OCI or PCI, or have approved mitigation plans 
in place. 

3. The SRB chair acts as an observer at any non-SRB-led activity to ensure 
his/her continued independence. 

4. The contractor must annually vet its contracted SRB members and 
consultants-to-the-board in compliance with the independence criteria 
outlined in Section 3.2 and Appendices C and D. The IPAO ensures 
integrity and compliance with this process. 
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Standing Review 
Board Overview 


T his chapter provides an overview of the Standing Review Board (SRB) 
and its purpose, function, and participation in the Life- Cycle Review 
(LCR) process. It defines the SRB’s role and explains the importance of its 
standing nature as well as lists the SRB’s stakeholders and the role of its 
Decision Authority. 


2.1 SRB Introduction 


NPR 7120.5, NASA Space Flight Program and Project Management Require- 
ments defines the life cycles of the space flight programs and projects with 
each life-cycle phase including one or more LCRs. An LCR comprises an 
independent review that provides an assessment of a program’s or project’s 
technical and programmatic status and health at a key point in its life cycle. 
NPR 7120.5 requires the use of a single, independent review team called the 
SRB to conduct certain LCRs. LCRs are essential to conducting, managing, 
evaluating, and approving space flight programs and projects, and are an 
important part of NASA’s system of checks and balances. NASA accords 
special importance to maintaining the integrity of its independent review 
process. LCRs provide the program or project and NASA’s senior manage- 
ment with a credible, objective assessment of the program’s or project’s 
progress, issues, risks, and status. An LCR is complete when the governing 
Decision Authority makes his or her decision to authorize a program or 
project to continue down the life cycle. 


The SRB process integrates the review requirements of NPR 7120.5, NPR 
7123.1, NASA Systems Engineering Processes and Requirements, the Mission 
Directorate, and the Center into a single LCR set of requirements. The SRB 
is responsible for fulfilling all the review requirements of all the Convening 
Authorities at each LCR, avoiding an individual review for each Convening 
Authority. 


The SRB process integrates the 
review requirements of NPR 7120.5, 
NPR 7123.1, the Mission Directorate, 
and the Center into a single LCR set 
of requirements. 
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2.2 SRB Review Criteria and Maturity States 


SRBs serve an advisory role to the 
Convening Authorities and have no 
programmatic or technical authority 
over the programs or projects. 


The Agency has six assessment 

criteria for all space flight programs 

and projects: 

• Alignment with and contribution to 
Agency strategic goals. 

• Adequacy of management 
approach. 

• Adequacy of technical approach. 

• Adequacy of the integrated cost 
and schedule estimates and 
funding strategy. 

• Adequacy and availability of 
resources other than budget. 

• Adequacy of the risk management 
approach. 


The standing nature of SRBs provides a strong advantage in terms of conti- 
nuity and familiarity with the program’s or project’s purpose, history, 
programmatic and technical approach, challenges, risks, and issues. The 
SRB process ensures that the program or project, Convening Authorities, 
Technical Authorities, and other appropriate stakeholders are briefed in a 
consistent fashion on results and conclusions based on the same material 
gathered by the same SRB. 

SRBs engage with the programs or projects around their assigned LCRs and 
normally are inactive between LCRs except as needed to maintain some 
level of awareness of the activities of the program or project. Members may 
attend program or project subsystem reviews as authorized by the SRB chair, 
the Review Manager, and the program or project. 

SRBs serve an advisory role to the Convening Authorities; consequently, 
they have no programmatic or technical authority over the programs or 
projects they review. SRBs present their findings and recommendations for 
consideration by the Convening Authorities. 


2.2 SRB Review Criteria and Maturity States 

NASA formulates programs and projects to implement a diversity of prod- 
ucts with widely varying costs and risks. For this reason, the SRBs have 
varying levels of assessment, participation, and reporting based on the cate- 
gorization of the program or project. 

NPR 7120.5 lists assessment criteria for all space flight programs and proj- 
ects with LCR entry/exit criteria per NPR 7123.1. The SRBs use these criteria, 
customized for each type of program implementation and for each LCR, 
in support of their independent assessment. Section 5.1 contains a detailed 
description of these criteria and their application. 

Appendix I of NPR 7120. 5E defines the expected maturity of program and 
project products and control plans at each LCR. Programs and projects are 
expected to have achieved these maturities unless the requirements for them 
have been tailored and approved. Appendix D of the NASA Space Flight 
Program and Project Management Handbook (PM Handbook) contains addi- 
tional information on maturity states. 


2.3 SRB Program and Project Reviews 

There are four basic types of programs: uncoupled, loosely coupled, tightly 
coupled, and single-project. The PM Handbook provides more detail on 
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2.4 SRB Participation in Selected Program or Project Internal Reviews 


the program types. There is a specific life cycle for each of these programs 
and a specific expected project maturity state for each review described in 
NPR 7120.5. 

For tightly coupled programs and their projects, the SRB can be either 
a single SRB for the program and all projects or separate SRBs for the 
program and each of the projects. Tightly coupled program reviews typically 
occur after all of the program’s projects have completed equivalent reviews 
at the project level. SRB participation in the LCR of each type of program 
and project is summarized in Table 2-1 . The Key Decision Points (KDPs) are 
shown in blue, and the level of board participation is indicated in the table. 


Table 2-1 Agency-Level Reviews Conducted by SRBs 


Review 

Uncoupled or 
Loosely Coupled 
Programs 

Single-Project 

Programs 

Tightly Coupled 
Programs 

Projects 

System Requirements Review (SRR) 

X 

X 

X 

X 

System Definition Review (SDR) , or 
Mission Definition Review (MDR) 

X 

X 

X 

X 

Preliminary Design Review (PDR) 


X 

X 

X 

Critical Design Review (CDR) 


X 

X 

X 

System Integration Review (SIR) 


X 

X 

X 

Operational Readiness Review (ORR) 


X 

X 

X 

Program Implementation Reviews (PIR) 

X 

X 

X 



Some reviews are only conducted at the request of the Mission Directorate 
Associate Administrator, the Center Director, or the Decision Authority. 
The Convening Authorities can also authorize the SRB to conduct special 
reviews as needed. Section 4.11 addresses special reviews. 


2.4 SRB Participation in Selected Program or 
Project Internal Reviews 

In coordination with the program or project manager, the SRB chair and 
selected SRB members may participate at program or project planned 
internal reviews as observers. Any SRB member, except the SRB chair, 
may serve as a member of the internal review board. The SRB chair is not 
permitted to be a member so as to preserve his/her independence. 


The SRB chair and some SRB 
members or consultants-to-the- 
board may attend program or project 
internal reviews as observers to gain 
insight into the program or project 
status and health. The program or 
project manager must approve their 
attendance. 
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3 Forming a Standing 

Review Board 


T his chapter provides a number of principles to consider when forming a 
Standing Review Board (SRB). The factors considered for membership 
are competency, currency, and independence. There is no master formula 
or predetermination in staffing boards (see Section 3.3 ) since each SRB is 
structured to fit the unique characteristics of the program or project under 
review. 


3.1 Structure 

NASA implements three SRB structures for Agency-level space flight 
program or project Life-Cycle Reviews (LCRs). They are the Civil Service 
Consensus Board (CS), the Civil Service Consensus Board with Expert 
Support (CS2), and the Non-Consensus Mixed Board (NC). SRB organi- 
zation, management, and reporting differ among these three structures. 

Each SRB has a single chair and a NASA Review Manager. 1 The table in 
Appendix E compares the features of the different SRB structures and 
provides detail to assist in board type selection. 

NASA prefers CS or CS2 boards, as civil service members are generally more 
current on Agency policy, procedures, and culture. Experience demonstrates 
that a consensus board leads to a more meaningful discussion of the review 
findings and recommendations, especially where dissenting opinions are 
discussed. NC boards are typically used when the required expertise of a 
member cannot be obtained from the civil service workforce. 


1 The NASA Review Manager may be from the Jet Propulsion Laboratory. 
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The SRB has three options for its 
structure. These options are the civil 
service consensus board, the civil 
service consensus board with expert 
support, and the non-consensus 
mixed board. A consensus board is 
preferred. 


3.2 SRB Independence and Integrity 


SRB members must be free of 
personal or organizational or 
positional conflict of interest. 


3.2 SRB Independence and Integrity 

SRBs must conduct assessments free of bias through a membership balanced 
in terms of knowledge, experience, and perspectives. Balanced unbiased 
boards fulfill NASA policy that seeks to ensure the integrity of SRBs. 

Balanced SRBs composed of highly qualified members and consultants-to- 
the-board from various sectors of society (i.e., academia, industry, govern- 
ment, and nonprofit organizations) enable NASA to produce accurate and 
objective assessments of its programs and projects consistently. 

NASA requires conflict-of-interest-free SRB members and consultants-to- 
the-board throughout the SRB process. Members and consultants-to-the- 
board must stay free from conflicts that have the potential to significantly 
impair their individual objectivity or create an unfair competitive advan- 
tage for any person or organization. The NASA policy guidance on conflict 
of interest is set forth in Appendix C . Appendix D contains a copy of the 
NASA forms for Background Information, Confidential Conflict of Interest 
Disclosure, and Non-Disclosure Agreement (NDA) that all non-civil service 
members who serve on an SRB must complete . 2 All non-civil service SRB 
members and consultants-to-the-board must provide a signed NDA and 
certified Confidential Conflict of Interest Disclosure before participating in 
any SRB activity. The contractor through which the services of the member 
or consultant-to-the-board are obtained will ensure the member or consul- 
tant-to-the-board has no conflicts of interest. The contractor will submit any 
mitigation concerns to the Contracting Officer for approval. NDA and the 
conflict of interest forms must be completed and signed. 

Conflicts of interest may include: 

• Personal conflict of interest based on the personal financial interests of 
the individual. 

• Organizational conflict of interest based on the interests of the individu- 
al’s employer. 

• Positional conflict of interest based on the position the civil servant 
holds. 


2 The NDA limits the individual’s use and disclosure of restricted information 
obtained during the course of SRB activities. These restrictions do not apply to 
information once it becomes publicly available. 
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3.2 SRB Independence and Integrity 


3.2.1 Civil Servant Conflict of Interest and Independence 
Screening 

Internal screening is performed to ensure the independence of civil servants 
on an SRB. All civil servants must have a current Office of Government 
Ethics Form 450 or Standard Form 278, as applicable, on file with NASA (or 
available to NASA) prior to being considered for SRB membership. These 
forms must be updated annually. 

The Langley Research Center (LaRC) Office of Chief Counsel (OCC) will 
identify disqualifying personal and positional conflicts of interest in accor- 
dance with the relevant laws and regulations governing standards of ethical 
conduct. 3 A civil servant must not participate in any SRB activity until the 
LaRC OCC has made a determination that the civil servant has no financial 
interests that will create a conflict with service on an SRB. When the OCC 
informs the Independent Program Assessment Office (IPAO) that a person 
cannot serve on the SRB due to a personal or positional conflict of interest, 
the IPAO may: 

• Find an alternative SRB member, 

• Request divestiture of a financial interest that creates the conflict of 
interest, or 

• Pursue a waiver for the disqualified individual. 

If a Mission Directorate, SRB chair, Center, or the IPAO seeks to pursue a 
divestiture or waiver, the IPAO must coordinate the action with the LaRC 
OCC or the Center must coordinate with the local OCC. In the event that a 
conflict of interest exists for a particular proposed civil service SRB member, 
as part of the waiver request, the Decision Authority may prepare a written 
statement explaining that an SRB’s need for a civil servant’s expertise and 
the importance of his/her participation on the SRB outweigh any concern 
that the member’s financial interest is so significant that it will call into 
question the integrity of the employee’s service on the SRB and Government 
operations. The statement may be appended to a request for an approved 
statutory waiver of the ethics prohibition (prohibiting participation on 
a matter in which the civil servant has a conflicting financial interest) 
submitted through the appropriate chain of authority in accordance with 
NPR 1900. 3B, Ethics Program Management, Paragraph 3.4.3. The appropriate 
authority must submit the waiver request to the NASA Office of General 
Counsel for concurrence and then to the Administrator for final signed 
approval before the civil servant participates in any SRB activity. 


3 See 18 USC § 208 and “Standards of Ethical Conduct for Employees of the 
Executive Branch” contained in 5 CFR (Code of Federal Regulation ) part 2635, as 
supplemented by 14 CFR 1207. 
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3.2.2 Contractor Conflict of Interest Screening 

To the extent consistent with the contractual requirements, the Contracting 
Officer (CO) on the relevant contract is responsible for facilitating the 
screening of any proposed contractor SRB member or consultant-to-the- 
board for organizational and personal conflicts of interest prior to initiating 
any work on SRB activities. The CO will conduct an organizational conflict 
of interest analysis in accordance with the Federal Acquisition Regulation 
(FAR), contract requirements, and Appendices C and D . 4 The CO is also 
responsible for ensuring organizational conflicts of interest are eliminated 
and mitigated (e.g., through recusal, limitation of future contracting, fire- 
walls, and NDAs). 

The responsible CO will coordinate with the local OCC to evaluate 
contractor personal conflict of interest concerns and make recommenda- 
tions on all issues . 5 When the local OCC recommends that an individual 
contractor employee or consultant not serve on an SRB due to a personal 
conflict of interest, the IPAO will coordinate with the CO to: 

• Request an alternative individual, 

• Inquire as to a possible divestiture of the conflicting interest, or 

• Pursue a personal conflict of interest waiver for the contractor employee 
or consultant . 6 

Personal and organizational conflict of interest analysis will be conducted 
annually. 

3.2.3 Positional Conflicts of Interest 

Civil servant SRB members must also remain free of positional conflicts of 
interest. 

A civil servant employed by an organization that institutionally supports 
the program or project (e.g., a NASA Center, Mission Directorate) may not 
serve as a member of an SRB unless it has been determined by the OCC, in 


4 The FAR provisions on organizational conflict of interest only apply to contrac- 
tors and consultants on an SRB. Those organizational conflict of interest provisions 
concerned with bias are designed in part to ensure the objectivity of any contractor 
or consultant on an SRB. 

5 For the Jet Propulsion Laboratory and the Advanced Physics Laboratory, the 
NASA Fieadquarters legal office is used. 

6 Under the NASA Policy Guidance on Standing Review Board Composition, 
Balance, and Conflicts of Interest (see Appendix C L the Decision Authority has the 
authority to approve a written determination that a contractor’s expertise outweighs 
the conflict of interest in those cases where the local OCC determines a personal 
conflict of interest exists. 
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consultation with the IPAO, that a positional conflict of interest does not 
exist as a result of the following requirements being met: 

• The service of the individual on the SRB must be based upon the unique 
scientific, technical, or programmatic expertise that the individual brings 
to the SRB; 

• The individual and the individual’s supervisory chain must not be located 
within the chain of command for programmatic-level decisions made at 
the program or project level; and 

• There must be a specific determination, made by the OCC, during the 
SRB appointment process that service by the individual will not compro- 
mise the independence or objectivity of the review. 


Having SRB members and 
consultants-to-the-board who have 
no conflicts of interest is mandatory 
to maintaining the independence of 
the assessment. 


3.3 Composition and Balance 

For a balanced SRB, the needs of the Convening Authorities and other 
stakeholders are considered. Some of their needs are unique to individual 
organizations, while others are shared needs. The selection and vetting 
process ensures the technical and programmatic areas are covered expertly 
and adequately, while simultaneously satisfying the Agency-level need to 
have an informed, independent assessment and recommendation to the 
Convening Authorities and Decision Authority at Key Decision Points 
(KDPs). In cases of reimbursable programs and projects, the SRB composi- 
tion will be determined based on the NASA-to-sponsor agreements for the 
work being performed. 

Members and consultants-to-the-board can be selected both from within 
the Agency and from external sources, including such communities as the 
Department of Defense, industry, academia, and other Government Agen- 
cies. When looking internally within the Agency, consideration is given to 
unique insights of the various NASA Centers and the perspective that cross- 
mission opportunities can add to SRB expertise. 

Depth is the degree of competency in a particular discipline or area and is 
a prerequisite for being selected for the SRB. However, competency is also 
viewed from management, programmatic, testing, and integration perspec- 
tives. As a program or project matures toward System Integration Review 
(SIR) and Operational Readiness Review (ORR), a best practice is to stream- 
line SRB participation by selectively using only the needed skill mix from 
its member pool for the ORR. On the other hand, an expert with rare and 
unique skills could be brought in to serve as a consultant-to-the-board for 
the SRB in a specific review only. SRBs that have members with breadth of 
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The factors considered for SRB 
membership are competency, 
currency, and independence. 


knowledge and experience have the advantage of topics being assessed by 
several individuals, resulting in a more thorough evaluation. 

Two additional attributes are independence and currency as a practitioner. 

In NASA, where technology, process, and policy are changing rapidly, 
currency is an important aspect to consider for a reviewer. Hence, in the 
selection of well-qualified SRB members and consultants-to-the-board, 
currency is a key consideration. 

3.3.1 SRB Membership Balance Assessment 

The SRB chair and Review Manager develop an SRB membership balance 
assessment, which may be presented to IPAO management and the 
Convening Authorities as required/requested. The balance assessment 
addresses affiliation, primary expertise, currency, competency, and inde- 
pendence. The assessment addresses the members’ demographics, such 
as industry versus civil servant participation, total NASA participation, 
NASA host Center’s versus other NASA Centers’ participation, and partici- 
pation from other Agencies. A skills matrix, presenting each member’s 
primary skill and secondary skills to be used as the basis for SRB selection, 
is compiled and is part of the balance assessment. The balance assessment is 
an important set of information used by the Convening Authorities in deter- 
mining the acceptability of the SRB membership. 

3.3.2 Size and Composition 

For an Agency-level review, the SRB chair. Review Manager, and lead 
Programmatic Analysis Group analyst are board members. When forming 
the SRB, a very important aspect is determining the “right size” of the 
membership that is able to meet the expectations of the LCR. Minimizing 
the number of members is considered best practice; however, every SRB size 
decision requires consideration of variables including balance, competency, 
currency, and relevance. The balance assessment documents the rationale 
for the board size and composition. 

The members are selected for the duration of the program or project life 
cycle. Multiple disciplines can be covered by one member (e.g., electrical and 
systems engineering). Consultants-to-the-board can be added temporarily 
to review specific items identified by the SRB members. 

There are many Mission Support Offices internal to the Agency that are 
defined by the Agency governance model to be independent of the program 
or project. These Mission Support Offices can give an SRB a second level of 
support when analysis is needed. Such support consultants-to-the-board 


CHAPTER 3. FORMING A STANDING REVIEW BOARD 


3.4 Selection and Approval of SRB Members and Consultants-to-the-Board 


can come from the Office of Safety and Mission Assurance, the NASA Safety 
Center, Center Safety and Mission Assurance organizations, the Office of 
the Chief Engineer, the Office of the Chief Health and Medical Officer, the 
NASA Engineering and Safety Center, and Center engineering organiza- 
tions. Another option to leverage existing resources is to use membership 
from other related teams; e.g., project SRB chairs may have membership on 
program SRBs. 


3.4 Selection and Approval of SRB Members and 
Consultants-to-the-Board 


SRB formulation includes the identification and approval of the SRB chair 
and all other board members and consultants-to-the-board, assignment of 
the Review Manager, and development of the Terms of Reference (ToR). (See 
Figure 3-1 .1 The ToR is the official document for final approval of the SRB 
members, consultants-to-the-board, SRB chair, and Review Manager. 


The Convening Authorities approve 
all SRB members and consultants-to- 
the-board. 


3.4.1 SRB Chair 

The SRB chair and the Review Manager of the SRB are the first members 
approved. 


3.4.1.1 Nomination 


The SRB chair is typically a leader who is also a recognized expert with 
relevant experience for the respective space flight program and project 
LCRs. In general, good communication skills (both written and oral) and 
time commitment are also desirable for leading all the required program or 
project LCRs. When possible, civil servant chair candidates are nominated 
as a best practice. 

3.4.1. 2 Approval 


The SRB chair and the Review 
Manager are the first SRB members 
approved. 


The SRB chair nomination requires collaboration among the Convening 
Authorities. The nomination can come from any of the Convening Authori- 
ties. Usually, more than one candidate is considered. All Convening Author- 
ities have the opportunity to review the nomination(s) and submit alterna- 
tive nomination(s). The Review Manager facilitates the identification and 
evaluation process of the candidates with the Convening Authorities until 
one candidate is found suitable to all of the Convening Authorities. If the 
Convening Authorities cannot reach agreement, the Decision Authority 
makes the selection. 
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'The Chief Engineer is not a Convening Authority for Category 3 projects. 

2 The Mission Directorate Associate Administrator acts as a Convening Authority only when not already acting as the Decision 
Authority. 

3 The Director of the Office of Evaluation is not a Convening Authority for Category 3 projects and Category 2 projects of less than 
$250 million. 

4 When applicable and at the request of the Office of the Chief Engineer, the Office of the Chief Health and Medical Officer/Health 
and Medical Technical Authority will determine the need for health and medical participation on the SRB. 

5 Terms of reference content may vary with the organization responsible for the SRB. 

6 For each life-cycle review conducted by an SRB, the SRB chair selects SRB participants from the approved list. 

Figure 3-1 Forming an SRB 


For Agency-level reviews, the SRB chair nomination is facilitated by the 
IPAO with the Convening Authorities. The IPAO will then process the 
approved nominee through legal and procurement to complete the neces- 
sary steps for bringing the nominee on board. The IPAO will then send a 
formal approval letter to the Convening Authorities for their electronic 
signatures to complete the nomination process. 

3.4.2 Review Manager 

3.4.2.1 Nomination 

The Review Manager performs the critical function of ensuring appropriate 
and consistent implementation of Agency policy, process, and products for 
LCRs conducted by an SRB. The Review Manager must possess a high level 
of knowledge of the SRB policies derived from NPD 1000.5, Policy for NASA 
Acquisition, NPR 7120.5, NASA Space Flight Program and Project Manage- 
ment Requirements, and NPR 7123.1, NASA Systems Engineering Processes 
and Requirements; and program and project review processes defined in the 
NASA Space Flight Program and Project Management Handbook, NASA/ 
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SP-2007-6105, NASA Systems Engineering Handbook, and this handbook. 

The Review Manager may serve on the SRB as a discipline expert. 

3.4. 2. 2 Approval 

The IPAO is responsible for assigning a Review Manager for Agency-level 
reviews. The same approval mechanisms are used for the Review Manager as 
are used to approve the SRB chair. 

3.4.3 SRB Members and Consultants-to-the-Board 


The Review Manager performs 
the critical function of ensuring 
appropriate and consistent 
implementation of Agency policy, 
process, and products for LCRs 
conducted by an SRB. 


3.4.3.1 Nomination 

The SRB members and consultants-to-the-board nomination process 
requires collaboration among the Mission Directorate Associate Adminis- 
trator, the NASA Chief Engineer, the Center Director, and the Director of 
the Office of Evaluation in accordance with NPR 7120.5. The SRB chair and 
Review Manager, in collaboration with the program or project manager and 
the program executive, initiate the nomination process for SRB members 
and consultants-to-the-board. The process starts with the SRB chair and 
Review Manager, in collaboration with those listed above, developing a list 
of required areas of content that the SRB will review over the full life cycle 
of the program or project and then developing a list of candidates to support 
all LCR needs. The Convening Authorities approve the list of participants. 

A good practice is to start with the program’s or project’s Work Breakdown 
Structure (WBS). Particular consideration should be given to the program’s 
or project’s risk areas. When considering subject matter experts to support 
the SRB, the best-case scenario for optimizing team size would be to have 
candidates who can support more than one skill area in addition to their 
primary area of expertise. It is also desirable to have institutional/functional 
support organization candidates from both non-host Centers and the host 
Center. 

For Agency-level reviews, the IPAO would be able to reference its subject 
matter expert database to expedite the search for qualified candidates who 
would meet the necessary requirements. 

3.4. 3. 2 Approval 


The nomination process develops 
a list of members and consultants- 
to-the-board needed for all LCRs 
conducted by the SRB over the 
programs' and projects' life cycle. 
The SRB chair selects participants 
from this list for each specific LCR. 


The SRB chair, working with the Review Manager, program or project 
manager, and program executive, develops the initial candidate list for the 
SRB. The SRB chair ensures that the proposed SRB has the appropriate 
balance relative to currency and competency. The Review Manager will 
facilitate the approval of the proposed SRB candidates by the Convening 
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The Convening Authorities 
must approve any change in the 
membership of the SRB. 

Authorities. If agreement cannot be reached among the Convening Authori- 
ties, the Decision Authority will make the final decision. 

Once a decision is reached, the candidate names are included in the ToR 
with the required supporting information. The candidates are approved 
when the Convening Authorities sign the ToR. 

If approval of the members is needed before approval of the ToR, the Review 
Manager will use an approval letter. The SRB approval letter contains the 
following as a minimum: program or project identification information, 
subject matter experts’ brief but relevant biography, appropriate organiza- 
tional/personal conflict of interest compliance verification statement, and a 
summary SRB skills matrix. The Review Manager will facilitate the approval 
of this letter by the Convening Authorities. 

The SRB chair should sustain a core body of members who participate in 
each TCR to provide continuity over the full program or project life cycle. 

As an LCR approaches, the SRB chair selects members and consultants-to- 
the-board whom he/she determines are needed to support that LCR. Since 
the participants are preapproved, the SRB chair is only required to notify 
the Convening Authorities of those selected prior to that LCR. 

3.4.4 Change Process for SRB Membership 

For Agency-level reviews, replacement of the Review Manager and of the 
Programmatic Analysis Group analyst will only need the Office of Eval- 
uation Director’s approval before distributing the results to the other 
Convening Authorities as updated information. 

Replacement of the SRB chair, members, and consultants-to-the-board can 
be approved and documented in two ways. The first way is to update the ToR 
with the changes if modifications to other parts of the ToR are required. The 
second way is to use a change letter when modifications of other parts of the 
ToR are not needed. 

The following information is included for either path: program or project 
identification information, subject matter experts’ brief but relevant biog- 
raphy, appropriate organizational/personal conflict of interest compli- 
ance verification statement, and a summary SRB skills matrix showing the 
changes. 
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Life-Cycle Review 
Process 


L ife- Cycle Reviews (LCRs) are conducted under documented Agency 
review processes. The NASA Space Flight Program and Project Manage- 
ment Handbook (PM Handbook) provides more detail on the LCR process. 
The LCR process provides: 


• The program or project with a credible, objective assessment of how it is 
performing. 

• NASA senior management with an understanding of whether: 

• The program or project is on track to meet objectives, 

• The program or project is performing according to plan, and 

• Impediments to program or project success are addressed. 

• A credible basis for the Decision Authority to approve or disapprove the 
transition of the program or project at a Key Decision Point (KDP) to the 
next life-cycle phase. 

The Standing Review Board (SRB) is only responsible for conducting inde- 
pendent reviews during specific LCRs. This chapter focuses on the LCRs 
conducted by an SRB. An integrated perspective of the overall review 
process is presented in Figure 4-1 . Figures 4-2 and 4-3 show key elements 
that are part of the LCR. 


The SRB is a required part of the 
Agency's LCR process. The SRB 
conducts only the specific LCRs 
identified in NPR 7120.5, NASA 
Space Flight Program and Project 
Management Requirements and 
as requested by the Convening 
Authorities. 


4.1 Terms of Reference 

The Terms of Reference (ToR) is the agreement between the SRB, Convening 
Authorities, and program or project that specifies the nature, scope, 
schedule, and ground rules for the conduct of the LCR by the SRB. Only one 
ToR is written for the life cycle of a program or project, and it includes all 
LCRs to be performed by the SRB. The ToR can be revised as necessary, but 
all revisions, additions, and deletions must be approved by the ToR signato- 
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Program/project 
initiates 
internal review 
process 


x Program/project conducts internal system/project reviews 
in accordance with approved review plan and Center practices; 
these internal reviews are typically the subsystem reviews for 
projects or integrated discipline and mission phase reviews 
for programs 


Program/project 
prepares summary 
package(s) defining the 
baseline, etc., for 
presentation at the LCR 
(e.g., SRR, SDR, PDR, CDR) 


SRB formation/ 
ToR development 1 


Program/project 
and SRB brief 
findings 


^Program/project 
dispositions SRB 
f indings 


Snapshot report 
to Decision 
Authority 


SRB reports out to 
program/project 
and coordinates 
snapshot report 
with program/ 
project 


Center Management 
Council 3 assessment 


Independent LCR 2 


May be one- or two-step review 

The 1st step of a 
two-step review 
is an assessment 
that focuses 
mainly on the 
technical 
content of the 
respective LCR 

The 2nd step of 
a two-step 
review is an 
assessment 
of all six Agency 
criteria 



Legend: r Program/project activity 

Acronyms: CDR = Critical Design Review, KDP = Key Decision Point, LCR = Life-Cycle Review, PDR = Preliminary Design Review, 
PMC = Program Management Council, SDR = System Definition Review, SRR = System Requirements Review, ToR = Terms of 
Reference. 

1 See Figure 3-1 . 

Successful readiness assessment prerequisite for advancing to the site review. See Section 4.2 for details. 

3 May be an Integrated Center Management Council when multiple Centers are involved. 

4 The Mission Directorate PMC is the Governing PMC for Category 2 and 3 projects. 

5 The Agency PMC is the Governing PMC for programs and Category 1 projects. 

6 The LCR is complete when the Governing PMC and the Decision Authority complete their assessment. 

Figure 4-1 Program/Project Independent Life-Cycle Review Process 
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ries. Appendices may be used to augment the original ToR to document new 
or unique requirements, unknown when originally written, for future LCRs. 

The SRB chair and the Review Manager lead the ToR development. They 
work collaboratively and iteratively with the Convening Authorities and 
the program or project to develop a ToR that meets the Agency’s assessment 
expectations. The Review Manager ensures that the ToR has been coordi- 
nated with all Convening Authorities’ points of contact and the program’s or 
project’s points of contact. Once the SRB chair, Review Manager, Convening 
Authorities’ points of contact, and program or project have an agreed-upon 
draft ToR, it is submitted to the Convening Authorities for approval. The 
ToR is developed before any LCR occurs. 

For tightly coupled programs, separate ToRs are not required for each 
project. The projects may be listed with the program under the description 
and governance section of the ToR. The program’s ToR may include the proj- 
ects’ LCRs. 

For loosely coupled or uncoupled programs, the projects typically have 
separate ToRs. For single-project programs, there will be a single ToR. 

The ToR template is provided in Appendix H . 


4.2 Readiness Assessment 

The readiness assessment is a check conducted to ensure that the program- 
matic and technical products for the LCR will be available with the expected 
maturity to support the LCR timelines. A successful readiness assessment 
is a prerequisite for the program’s or project’s advancing to the site review 
under the planned timeline. The content of the technical and programmatic 
products is not assessed by the SRB at this point. The Review Manager and 
the SRB chair work with the program or project to schedule an appropriate 
time for this assessment. 

A readiness assessment is typically conducted 30-90 days before the site 
review and can be accomplished via a teleconference between the SRB 
chair, the Review Manager, the Center Director (or designated Technical 
Authority), and the program or project manager. The Program Executive is 
invited. In this discussion, the readiness of the technical and programmatic 
products to support the requirements of the LCR under the planned time- 
lines is addressed. The assessment is made with respect to the LCR param- 
eters in Table 4-1 . In a two-step review process, there may be one or two 
readiness assessments. 
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4.2 Readiness Assessment 


The SRB chair provides an 
assessment of the program's or 
project's readiness to enter the LCR. 


4.3 Life-Cycle Review Methods 


Table 4-1 Maturity Parameters to Be Assessed 


Maturity Parameter 

Requirement Location 

Review entry criteria 

NPR 7123.1, Appendix G 

Review success criteria 

NPR 7123.1, Appendix G 

Control plans maturity matrix 

NPR 7120.5E, Appendix 1 

Products maturity matrix 

NPR 7120. 5E, Appendix 1 

Expected maturity state overall at KDP reviews and specific LCRs 

NPR 7120.5E, Tables 2-3-2-6 

Maturity tables (with review criteria details) 

NASA Space Flight Program and Project Management 
Handbook, Appendix D 



The SRB chair develops his/her individual assessment of the program’s or 
project’s readiness. If the SRB chair agrees that the program or project is at 
the proper programmatic and technical level to support the Agency’s matu- 
rity expectations for that LCR, the LCR is held. If the SRB chair’s assess- 
ment is not aligned with that of the program or project, the disagreement 
is reported to the Decision Authority, who determines whether to proceed 
with the LCR. 


4.3 Life-Cycle Review Methods 


All LCRs must assess both the program’s or project’s technical maturity 
and its alignment with the Agency’s six assessment criteria identified in 
NPR 7120.5, NASA Space Flight Program and Project Management Require- 
ments. Section 2, and described in Section 5.1 of this handbook. Reviews are 
conducted as either a one-step or a two-step review. NPR 7120.5 specifies 
that the program or project manager determine whether a one- or two-step 
review will be conducted. This decision is made well in advance of the LCR 
to provide time for the program or project and the SRB to prepare for the 
LCR. 

NPR 7120.5 requires the program or 
project manager to determine if the 
LCR is a one-step or two-step review. 

There are cases, particularly for human space flight programs and projects, 
where an internal program or project review is held concurrently with the 
SRB review. 


4.3.1 One-Step Review 


A one-step review is an LCR chaired by the SRB chair. All six Agency 
assessment criteria are reviewed in a one-step review. The one-step review 
is referred to by the name of the LCR. For example, the one -step review 
preceding KDP C is called the “Preliminary Design Review (PDR) Life- 
Cvcle Review.” Fisure 4-2 presents an overview of the one-step review usins 
the PDR as the example. 
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4.3 Life-Cycle Review Methods 



Acronyms: CMC = Center Management Council, DPMC = Division Program Management Council, JCL = Joint Confidence Level, 
KDP = Key Decision Point, LCR = Life-Cycle Review, PM = Program or Project Manager. 

Notes: A one- or two-step review may be used for any LCR. This handbook provides information on the readiness assessment, 
snapshot reports, and checkpoints associated with LCRs. Figure is not drawn to scale. 

Figure 4-2 One-Step PDR Life-Cycle Review Overview 


4.3.2 Two-Step Review 

Typically, a two-step review is appropriate when the program or project 
needs the results of the first step before it can fully mature the cost and 
schedule products for the integrated LCR (step two). When a two-step 
review method is used, the second step of the review covers all six of the 
Agency’s assessment criteria. 

The first step of the review typically focuses on the program’s or project’s 
technical maturity and health, taking into consideration preliminary cost 
and schedule information available at that time. This step addresses the 
adequacy of the program’s or project’s technical approach, as defined by 
NPR 7123.1, NASA Systems Engineering Processes and Requirements entrance 
and success criteria, and establishes the technical baseline. The first step of 
the two-step review is conducted by the SRB and chaired by the SRB chair. 

The second step of the review occurs no later than six months after the 
conclusion of the first step. It is an independent review conducted by the 
SRB and chaired by the SRB chair. Figure 4-3 presents an overview of the 
two-step review using the PDR as the example. 
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Acronyms: CMC = Center Management Council, DPMC = Division Program Management Council, JCL = Joint Confidence Level, 
KDP = Key Decision Point, LCR = Life-Cycle Review, PDR = Preliminary Design Review, PM = Program or Project Manager. 

Notes: A one- or two-step review may be used for any LCR. This handbook provides information on the readiness assessment, 
snapshot reports, and checkpoints associated with LCRs. Figure is not drawn to scale. 

Figure 4-3 Two-Step PDR Life-Cycle Review Overview 


4.3.3 Human Space Flight Review 

There are cases, particularly for human space flight programs and projects, 
where the program or project uses the internal LCR to make formal deci- 
sions to complete its technical work and align this work with the budget and 
schedule. In these cases, the program or project manager may co- chair the 
LCR with the SRB chair, since he/she is using this forum to make program 
or project decisions, and the SRB will conduct the independent assessment 
concurrently. The program or project manager works with the SRB chair 
to develop the LCR agenda and agree on how the LCR will be conducted to 
ensure that it enables the SRB to fully accomplish the independent assess- 
ment. The program or project manager and the SRB chair work together 
to ensure that the ToR reflects their agreements and that the Convening 
Authorities approve the approach. 
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4.4 Snapshot Report 

Rapid reporting to the Convening Authorities and Decision Authority is 
essential to efficient and effective management of programs and projects. 

To support this requirement, the SRB chair is required to provide a one- 
page written summary of his/her preliminary findings no later than 24 to 
48 hours after the site review (see Section 4.8 ) conclusion. This summary 
report is known as the snapshot report. 

The snapshot report contains an LCR overview, the SRB’s summary find- 
ings, a discussion of significant issues, a discussion of significant risks, and 
the schedule for briefing all required management councils that will lead up 
to the applicable Governing Program Management Council. 

The SRB chair briefs the Convening Authorities on the snapshot informa- 
tion in a teleconference setting after the site review’s conclusion. The SRB 
chair provides the program or project manager with a courtesy copy of the 
snapshot report prior to the teleconference. 

For a one-step review process, one snapshot report is required for the single 
review, as shown in Figure 4-2 . For the two-step review process, a snapshot 
report is required after both the first step and the second step, as shown in 
Figure 4-3 . 


4.5 Checkpoints 

Neither the SRB nor the SRB chair is directly involved in the checkpoint 
process. Any involvement by the SRB in checkpoints will be as directed by 
the Decision Authority. 

At a checkpoint, the program or project manager describes to the Decision 
Authority the detailed program or project plans for significant decisions, 
activities, and commitments. The Decision Authority provides the program 
or project with interim authorization, guidance, and direction. 


4.6 SRB Kick-Off Meeting 

The SRB kick-off meeting is a prepatory activity that precedes the active 
engagement of the SRB in the site review (see Section 4.8 ). An SRB kick- 
off meeting is conducted before each LCR the SRB conducts. The objec- 
tive of the meeting is to familiarize the SRB members and consultants-to- 
the-board with the current state of the program or project under review, 
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4.4 Snapshot Report 


The Snapshot Report, which is a one- 
page written summary of the SRB's 
preliminary findings, is provided 
within 24 to 48 hours of the site 
review's conclusion. 


4.6 SRB Kick-Off Meeting 


The SRB chair and the Review 
Manager conduct a kick-off meeting 
prior to each LCR to familiarize the 
SRB members and consultants-to- 
the-board with the current state 
of the program or project and 
the expectations of NASA senior 
management and the SRB chair. 


the current LCR process, any new policies, and the expectations of NASA 
management. 


4.6.1 Preparation for the Meeting 

The SRB chair and the Review Manager prepare the kick-off meeting 
agenda. Coordination and execution of the meeting is the responsibility 
of the Review Manager. The SRB members and consultants-to-the-board 
should participate in the meeting. It is recommended that the first kick-off 
meeting be conducted at a designated site. The program or project manager, 
the Program Executive, and the Strategic Investments Division (SID) analyst 
usually present during this meeting. Subsequent kick-off meetings for this 
SRB may then be conducted via WebEx or teleconference. The SRB members 
should prepare for the kick-off meeting by familiarizing themselves with the 
program or project and this handbook. The SRB chair will request a short 
briefing from the members who attended any program or project subsystem 
reviews prior to the kick-off meeting. In addition, the SRB program analyst 
will conduct a briefing for the SRB chair and Review Manager on best prac- 
tices and tools that are applicable to the specific review prior to the kick-off 
meeting. 

4.6.2 Meeting Attendees and Meeting Conduct 

For Agency-level reviews, the Mission Directorate Program Executive, the 
SID point of contact, and the Technical Authority are invited to the kick-off 
meeting. The Independent Program Assessment Office Director or designee 
typically briefs the SRB on his/her expectations and discusses the SRB process 
at a high level. The SID’s point of contact presents an overview of the budget 
formulation and external reporting status for the program or project and the 
financial portfolio of the mission for the program or project under review. 

The SRB kick-off meeting is typically held 30-90 days prior to the start of 
the site review. This meeting provides the SRB with insight into Agency 
expectations, expectations of the SRB during the site review, and other 
topics deemed pertinent by the SRB chair. During the meeting, the SRB 
program analyst coordinates with the SRB regarding the SRB risk input 
process, risk meetings, and preliminary risk analysis pertinent to the review 
and the site visit. Also during the kick-off meeting, the SRB chair and 
Review Manager share content from the SRB Document Library, facilitate 
program or project document access, and ensure the initiation of the appro- 
priate independent programmatic analyses. This early interaction eliminates 
the need for many informational questions asked by SRB members during 
the time-constrained LCR. 
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4.7 Programmatic Data Submittal from Program or Project 


4.7 Programmatic Data Submittal from Program 
or Project 

It is very beneficial for the program or project management to meet with the 
chair, the Review Manager, and the lead programmatic analyst at the SRB 
planning session to plan for the review approximately six months in advance 
of the site review. Requirements for the review, required data products, and 
the SRB review timeline should be discussed and understood by all parties. 
This will provide a basis of expectations between the program or project and 
the SRB, as well as allow for a more streamlined data delivery process. 

A component in all reviews is the programmatic assessment of the 
program’s or project’s progress relative to the schedule and cost. The SRB 
programmatic analysis is performed prior to the site review. Analysis results 
are briefed out to SRB members and consultants-to-the-board at the time of 
the site review. If there are any major discrepancies, an opportunity to revise 
the analysis is available during the SRB caucus sessions. 

Before the site review, data access and then two data deliveries of program- 
matic data are required to allow for preparatory analyses by the SRB. The 
following data access and release timelines were established using best prac- 
tices to meet the SRB briefing schedule. During the SRB planning session 
with the programs or projects, and well in advance of the first programmatic 
access, delivery timelines maybe adjusted if agreed to by all parties. 

The SRB programmatic analysis is 
performed prior to the site review. 

Access to existing programmatic data commences 100 days prior to the 
site review. There is no intention for the projects to do additional work 
preparing the products available for data access at 100 days before the site 
review. It is recognized that the data are preliminary and products may 
be incomplete and very likely could change before subsequent data deliv- 
eries and/or the review. At that time, data are posted on the SRB website 

Access to existing programmatic 
data commences 1 00 days prior to 
the site review. This allows the SRB 

for the SRB to review. Requirements for the data products are outlined in 

members to become familiar with 

NPR 7120.5: the Program Plan; the risk list; staffing requirements and plans; 
infrastructure requirements; the Integrated Master Schedule (IMS); Basis 
Of Estimates (BOEs) for both cost and schedule; documentation of perfor- 
mance against plan; plan for work to be accomplished during implementa- 
tion; external cost and schedule commitments; Cost Analysis Data Require- 
ment (CADRe) data; Technical, Schedule, and Cost Control Plans; the Risk 
Management Plan; the Work Breakdown Structure (WBS), the time-phased 
budget allocation by WBS, and WBS dictionary; Cost (Unallocated Future 
Expenses — UFE) and schedule reserves and the basis for reserves; and 
Earned Value Management (EVM) data for contracts requirement EVM. 
Additionally, other documents that are useful include: past status reports; 

the program or project prior to 
participating in subsystem reviews 
and to communicate any issues to the 
program or project in advance. 
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4.8 Site Review 


Data delivery is made 60 days prior 
to the site review. 

the Master Equipment List (MEL) and mass properties report, the Power 
Estimate List (PEL), and metrics for software lines of code. Depending upon 
the LCR, it is understood that, in many cases, these data may be in prelimi- 
nary format. Access to data 100 days prior to the review allows the SRB 
members to become familiar with the program or project prior to partici- 
pating in subsystem reviews, and communicate any issues to the program 
or project in advance; it also assists the chair in evaluating whether the 
program or project meets the criteria for the readiness assessment milestone. 

The first formal data delivery is made 60 days prior to the site review. At this 
time, if any of the aforementioned data products have been updated, those 
updates should be re-posted to the SRB website. Additionally, an updated 

An updated risk list; an updated IMS 
and an analysis schedule, if a range 
estimate or JCL analysis is being 
performed; and the cost estimate 
should be formally delivered. 

risk list; an updated IMS and an analysis schedule, if a range estimate or 
Joint Confidence Level (JCL) analysis is being performed; and the cost esti- 
mate should be formally delivered and posted to the website. The SRB will 
begin programmatic and risk analysis once these data are received. The SRB 
risk meetings begin upon the receipt of these data. It is understood that this 
is the initial data drop and that there could be changes (potentially signifi- 
cant) before the final analysis commencing 20 days prior to the site review 
when the final data deliveries are received. 

The final data delivery occurs 20 days 
prior to the site review. 

The final data delivery occurs 20 days prior to the site review. If any of the 
aforementioned data products have been updated, those updates should be 
posted to the SRB website. The final data delivery includes the final versions 

The final data delivery includes 
the final versions of the program 
or project risk list; the IMS and an 
analysis schedule, if a range estimate 
or JCL analysis is being performed; 
and the final cost estimate. This 

of the program or project risk list; the IMS and an analysis schedule, if a 
range estimate or JCL analysis is being performed; and the final cost esti- 
mate. 

This delivery supports the final SRB risk evaluation meeting prior to the 
site review. The SRB program analysts will provide analysis results to the 
SRB prior to the site review. Because the data access and first data delivery 

delivery supports the final SRB risk 
evaluation meeting prior to the site 

occur well before the review, the data and products are likely to change. The 
SRB should not view these changes as instability in the project but rather a 

review. 

normal part of the design maturation process. 

The site review is the formal, 
independent review of the programs 
or projects by the SRB for the LCR. 

4.8 Site Review 

The formal independent review of the program or project by the SRB is 
called the site review. The site review’s start and end dates are, respectively, 
the anchor points for the activities preceding and following the site review. 
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4.8 


4.8.1 Preparation for the Site Review 

The program or project develops the first draft of the site review agenda 
prior to the readiness assessment. Using this draft as a starting point, 
the final agenda is negotiated by the SRB chair, Engineering Technical 
Authority, the Safety and Mission Assurance Technical Authority, the 
Program Executive, the Center representative, the Review Manager, and 
the program or project. The final agenda is due no later than 30 days before 
the site review and must have the concurrence of the SRB chair, the Tech- 
nical Authority, and the program or project manager. The review schedule is 
developed with a goal of minimizing adverse impacts to the ongoing activi- 
ties of the program or project. 

The SRB chair is responsible for ensuring that the depth of the site review 
enables the SRB to make an informed assessment of whether the program’s 
or project’s state meets the LCR’s success criteria and the expected maturity 
states. 


4.8.2 Conducting the Site Review 

During the site review, the program or project presents its status through 
sequential briefings for each topic, typically given by the program or project 
leadership. The SRB chair presides over the review and is responsible for 
keeping it on schedule. The presenters answer questions from the SRB 
members in real time, if possible. If further detail is required, the program 
or project may offer to provide the necessary information later in the review 
or arrange a splinter session in parallel with additional presentations. 
During the site review, the SRB members may submit Requests For Action 
(RFAs). The RFAs must comply with the RFA process approved for the 
review. SRB members may sponsor RFAs from nonmembers. 

4.8.3 Reporting Out Site Review Findings 

SRB members and consultants-to-the-board determine the strengths and 
weaknesses of the program or project and report their findings on their 
Individual Member Independent Reports (IMIRs) and score cards. 

When the review concludes, the SRB meets to complete its assignments, and 
each member reports his/her findings and conclusions. Consultants-to- 
the-board provide their technical analyses, but do not participate in devel- 
oping the final SRB position. The SRB members and consultants-to-the-board 
provide the SRB chair and the Review Manager with an initial written IMIR 
prior to the program or project out-brief and a final IMIR within 48 hours of 
the site review’s conclusion. Section 5.5 provides more detail on the IMIR. 
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4.9 Key Decision Points 


The Decision Authority uses the SRB 
assessment and other information to 
make the decision for a program's or 
project's advancement to the next 
life-cycle phase. 


4.9 Key Decision Points 

Central to the program and project management process are the program 
and project life cycles and the KDPs within these life cycles. A KDP is an 
event where the Decision Authority determines the readiness of a program 
or project to advance to the next phase of the life cycle (with the excep- 
tion of KDP E; the program or project transitions to Phase E at the Post- 
Launch Assessment Review (PLAR)). Although the KDP is outside of the 
scope of the SRB’s responsibilities, the SRB provides essential information 
to the Decision Authority to make this determination. Understanding what 
information the Decision Authority needs to make the decision is critical in 
conducting an effective review. The standard needs and any special needs 
of the Decision Authority in support of the KDP must be understood and 
incorporated into the ToR. 


4.10 Late Life-Cycle Reviews 

The Operational Readiness Review (ORR) is the last TCR the SRB routinely 
conducts . 1 The ORR is conducted like any other LCR, except that the SRB 
chair reports the ORR’s results at the Mission Readiness Review (MRR), 
Mission Readiness Briefing (MRB)/Flight Readiness Review (FRR) for 
human space flight. 

After the ORR, all SRB members and consultants-to-the-board with the 
exception of the SRB chair and Review Manager will conclude their activi- 
ties. The SRB chair and the Review Manager are retained through launch. 

For supporting briefings after the ORR that lead to the KDP E, the SRB chair 
represents the SRB regarding the results of the ORR assessment. On other 
items, the SRB chair provides only his/her personal opinion/views since the 
SRB has been dissolved by this point (unless it is reconstituted at the request 
of the Convening Authorities). 

The Centers generally conduct the LCRs after the ORR and do not use an 
SRB. These reviews happen in rapid succession and include the Safety and 
Mission Success Review (SMSR), the FRR (for human space flight, the FRR 
is usually chaired by the Mission Directorate Associate Administrator), the 
PLAR, and the Critical Events Readiness Review (CERR). Center practices 
are followed for Center- convened reviews. The Center is responsible for 
assembling the review team. The Center must procure (contract and fund) 


'For programs, the SRB is usually retained to conduct the Program Implementa- 
tion Review (PIR). 
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any former SRB member that it desires to be on the institutional review 
team for post-ORR reviews. Such a member represents the institutional 
review team, not the SRB. 

The Convening Authorities may request that the SRB participate in or 
conduct any of these late LCRs. The SRB should know well in advance of the 
ORR if it will conduct any of the late LCRs. If the SRB is conducting reviews 
after the ORR, the approach for the execution, briefings, and written reports 
is streamlined for efficiency. Since each program or project is different with 
different timing for the late reviews, the planning for each review’s execu- 
tion, briefings, briefing content, and combination of review briefings is 
unique for each program or project. 


4.11 Special Reviews Conducted by the SRB 

Per NPR 7120.5, the Decision Authority can request the SRB to lead other 
LCRs or special reviews. A special review follows the same process, proce- 
dures, and requirements as an LCR, including reporting, unless a lesser set 
of requirements is defined in the sanctioning document. Additional outside 
experts may be used as needed. 

The special review focuses on a specific topic or set of issues. Circumstances 
that may warrant a special review include unanticipated changes to the 
program’s or project’s baseline; trends indicating the program or project is 
not meeting technical, cost, or schedule requirements; issues maturing an 
enabling technology; or other areas of special attention, such as earned value 
management. The Review Manager and SRB chair work with the autho- 
rizing Convening Authority to identify the issues to address, the execu- 
tion requirements, reporting requirements, and method for conduct of the 
review. The sanctioned governing document is typically a ToR or a Memo- 
randum of Understanding, which includes the reason for the special review 
and all conduct, assessment, and reporting requirements. The SRB chair 
and Review Manager either develop the governing document with the 
Convening Authority, or the Convening Authority provides this document 
to them. The Review Manager coordinates the required approvals, which are 
the authorizing Convening Authority and — typically — the Mission Direc- 
torate Program Executive, the program or project manager, and the SRB 
chair. 

4.11.1 Rebaseline Review 

An important special review is the rebaseline review. The three criteria for 
requiring a rebaseline review are in NPR 7120.5. The Decision Authority 
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4.1 1 Special Reviews Conducted by the SRB 


At the request of the Convening 
Authorities, the SRB can participate 
in LCRs after the ORR. 


The Decision Authority can request 
the SRB to lead other LCRs or special 
reviews. 


4.1 1 Special Reviews Conducted by the SRB 


decides if the SRB or another body conducts the rebaseline review. The 
review is conducted like a KDP C LCR using the requirements, content, and 
expectations for that review. The review revisits the maturity expectations 
that set the baseline at approval for implementation. 

Per NPR 7120.5, a rebaseline review requires tightly coupled programs, 
single-project programs, or projects with an estimated life-cycle cost greater 
than $250 million to provide a resource-loaded schedule and a cost estimate 
with basis of estimate, and to perform a risk-informed probabilistic analysis 
that produces a JCT. Loosely coupled and uncoupled programs are required 
to perform an analysis that provides a status of the program’s risk posture 
when a project’s Agency Baseline Commitment (ABC) is rebaselined. 
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5 Standing Review 

Board Products 


T he Standing Review Board (SRB) is charged with the responsibility of 
making an independent assessment of the program’s or project’s health 
and maturity. The SRB’s role is to provide the Convening Authorities with 
an expert judgment of the adequacy of the program’s or project’s technical 
and programmatic approach, risk posture, progress relative to the baseline, 
and readiness to advance to the next development level. 

An SRB has three primary functions: (1) to perform complete, comprehen- 
sive, and independent assessments of the program or project; (2) to develop 
findings and formulate recommendations based on these assessments; and 
(3) to report its results to the program or project and Convening Authorities. 

The following are key points regarding SRB responsibilities and products: 

• Depth of penetration. It is the responsibility of the SRB to establish 
a review level that sufficiently meets the requirements of the Terms 
of Reference (ToR) and enables the SRB to determine if the program 
or project is within the guidelines of its technical and programmatic 
requirements. SRB outputs are briefed to the program or project under 
review prior to being provided to NASA management. 

• SRB awareness between Life-Cycle Reviews (LCRs). Because the 
SRB is on standby between LCRs, it is the responsibility of the Review 
Manager to maintain contact with the program or project and coordi- 
nate with the SRB chair regarding the informational materials provided 
to SRB members outside of the LCRs. Examples of materials that may 
be provided to the SRB team are presentation material from periodic 
reviews — e.g., quarterly reviews, risk reviews, and major decisional 
change boards. The SRB members will not attend the program’s or proj- 
ect’s internal meetings or reviews outside of the LCRs, unless this is coor- 
dinated with the Review Manager, the SRB chair, and the program or 
project. 
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5.1 Assessment Criteria 


At the LCR, the SRB assesses the 
program's or project's health and 
status relative to the six assessment 
criteria identified in NPR 7120.5. 

The guidance for assessing these 
six criteria and technical success 
criteria is in NPR 7123.1 and the 
maturity matrices for control plans 
and products in NPR 7120.5 and the 
PM Handbook. 


• SRB ownership of programmatic analyses. The SRB has full owner- 
ship of the programmatic assessments because they link cost, schedule, 
and management with the technical aspects of the program or project. 
Programmatic assessments combined with the technical aspects formu- 
late a complete status of the program or project under review. 

• Time criticality for preparation and review of programmatic anal- 
yses. Programmatic data (as described in Section 4.7 ) must be received 
within the required timeframes in order to afford the SRB an opportu- 
nity to provide feedback to the program or project prior to a review. This 
provides the program or project with the opportunity to make any neces- 
sary data adjustments, as opposed to receiving a notification that the 
program or project does not meet requirements. 


5.1 Assessment Criteria 

LCR assessment criteria (discussed in Sections 5.1. 1-5. 1.6) are presented in 
NPR 7120.5, NASA Space Flight Program and Project Management Require- 
ments and are required for all LCRs. These criteria are used for all SRB- 
conducted LCRs in support of their independent assessment. The criteria 
are customized for each type of program implementation and each LCR. 
Using the same assessment criteria approach throughout the life cycle with 
emphasis consistent with the entrance and success criteria from NPR 7123.1, 
NASA Systems Engineering Processes and Requirements creates a consistent 
metric for traceability. The evaluation of the assessment criteria is supported 
by the maturity expectations of the control plans, products, and overall 
expected maturity state provided in matrix form in NPR 7120.5 and the 
NASA Space Flight Program and Project Management Handbook (PM Hand- 
book). The matrices are discussed in Section 5.2 . 

These assessment criteria are helpful in establishing the scope of SRB inde- 
pendent assessment activities and are used by the SRB to organize and 
summarize its findings (discussed in Section 5.4.2 ). 

The standard metric for the SRB success criteria evaluations is a three-level 
metric scale: i.e., successful (green), partially successful (yellow), or unsuc- 
cessful (red). This is sometimes referred to as a “stop-light” assessment. 

The SRB provides assessments for each of the Agency’s six criteria, along 
with a supporting rationale that addresses the assessment metrics provided 
as guidance in Tables 5-1 and 5-2 . As the program or project matures, the 
metrics for the criteria should become more demanding. A deficiency that 
might be acceptable early in the program’s or project’s life cycle is likely to 
be unacceptable later. It is up to the SRB to use its expertise to evaluate the 
program or project, taking into account life cycle stage or other circum- 
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5.1 


stances and assessing the risks that any deficiency poses against the “green” 
standard for successful execution of the program or project. Examples for a 
program and project are in Tables 5-1 and 5-2 . respectively. 

The assessment of the criteria should address the maturity expectations of 
the applicable milestone products and control plans in the maturity matrices 
in NPR 7120.5 and the expected maturity state table in the PM Handbook. 

5.1.1 Alignment With and Contributing to Agency Needs, 
Goals, and Objectives, and the Adequacy of Requirements 
Flow-Down From Those 

One of the first assessments each SRB must perform in the program or 
project life cycle is the alignment of program or project requirements with 
Agency needs, goals, and objectives, and determination of how well these 
requirements flow down to drive all defined levels of program content and 
project design without stray or open-ended requirements. This assess- 
ment typically takes place in the Formulation phase leading to the program 
System Requirements Review (SRR) and, for projects, may continue into 
Phase B as the project continues to refine the definition of its design at the 
subsystem and component levels. The System Requirements Document 
and Requirements Traceability Report are two key documents that the 
SRB should use in conducting this assessment. The SRB should complete 
its initial assessment findings before program acquisition or at the start of 
Phase B for a project. This alignment is also assessed at subsequent LCRs. 

5.1 .2 Adequacy of Management Approach 

The SRB will perform an evaluation of how well the program or project is 
managing its responsibilities. The scope of this evaluation includes (1) the 
management approach, e.g., organizational structure, integrated product 
teams, lines of authority; and (2) management processes and practices for 
planning, tracking, and control. An expected benefit of this SRB assessment 
is the contribution of lessons learned from the background of experience 
that a well- qualified SRB team can offer. 

5.1 .3 Adequacy of Technical Approach as Defined by 
NPR 71 23.1 

Technical assessments are somewhat different for projects and tightly 
coupled programs versus uncoupled or loosely coupled programs; therefore, 
each is addressed separately in Sections 5. 1.3.1 and 5. 1.3. 2 . 
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5.1 Assessment Criteria 


Table 5-1 Example of Program Assessment Guidance 


Criteria 

Program Assessment Metrics 

Successful 


Alignment 
with and 
contributing to 
Agency needs, 
goals, and 
objectives 

Program objectives are prioritized 
and well-aligned with strategic goals; 
objective-driven requirements are 
defined for current and near-term 
projects. 

Program objectives are not well-aligned 
with strategic goals; requirements for 
near-term projects are immature. 

Program objectives are notional and/ 
or do not align with strategic goals; 
requirements for existing projects may 
be lacking and do not exist for near- 
term projects. 

Adequacy of 
management 
approach 

The program organizational structure 
is defined and effective; interfaces to 
projects are clear; program policies 
and controls are defined; the program 
base is adequate. 

The program organizational structure 
lacks clarity; lines of authority may be 
duplicated; policies/controls are not 
well defined; interfaces are incomplete; 
program base is weak. 

Organizational structure is 
unacceptable; control processes are 
notional and not in place; necessary 
interfaces are not defined; program 
base is not defined. 

Adequacy 
of technical 
approach 

A 10-year architecture exists, 
consistent with program/Agency 
goals; project concepts exist for the 
architecture that are driving near-term 
technology investments; key external 
interfaces/needs are defined. 

The 10-year architecture is notional 
and not always consistent with Agency 
goals; future mission concepts are 
inadequate for planning guidance; 
external needs are poorly defined. 

A 10-year architecture does not exist; 
future mission concepts are without 
basis; little or no planning guidance 
exists for current readiness investments. 

Adequacy of 
the integrated 
cost and 
schedule 
estimate 
and funding 
strategy in 
accordance 
with 

NPD 1000.5 

The current program budget and 
phasing are adequate to support 
existing program scope; the approved 
5-year budget plan is sufficient to 
implement the Program Plan; the 
program funding wedge is adequate 
for the formulation of projects beyond 
the 5-year horizon; the project and 
program UFE is adequate to support 
the program JCL. 1 

The PIMS consisting of schedule 
data for all project effort included 
in the program scope and WBS 
and with all effort that is under 
the responsibility of the program 
organization to perform shall be 
delivered electronically, if logical 
relationships between projects exist, 
they are linked within the PIMS. The 
program and project critical paths 
are identifiable within the analysis/ 
PIMS. Schedule data for all effort that 
falls under the responsibility of the 
program should be reflected in lower 
discrete detailed schedules with 
vertical and horizontal integration in 
the PIMS. Program costs are mapped 
to the PIMS. The analysis/PIMS 
follows government best practices 
and is green on schedule health 
check. Program has and is following 
a program Schedule Management 
Handbook. The SRB is able to use 
the PIMS for risk identification and 
schedule and risk analysis of program 
risks. 

The current and approved 5-year 
baseline budget and phasing may not 
be adequate to support the Program 
Plan; the program funding wedge may 
not be adequate for the formulation 
of projects beyond the 5-year horizon; 
program and project UFE is either 
phased inappropriately or falls short of 
levels needed to support program and 
project JCLs.’ 

The program analysis schedule or 
PIMS consisting of schedule data 
for all project effort included in the 
program scope and WBS is partially 
identified, and with most of the effort 
that is under the responsibility of the 
program organization to perform 
shall be delivered electronically. The 
program and project critical paths 
are partially identifiable. If logical 
relationships between projects exist, 
they are partially linked within the 
PIMS. Schedule data for all effort that 
falls under the responsibility of the 
program should be reflected in lower 
discrete detailed schedules with vertical 
and horizontal integration in the PIMS. 
Program costs are partially mapped to 
the PIMS. The analysis/PIMS attempts to 
implement government best schedule 
practices, but is yellow on health check. 
The program has but is not following 
a Program Schedule Management 
Handbook. The SRB is able to use 
the PIMS for risk identification and 
preliminary schedule and risk analysis of 
program risks. 

The current program budget and 
phasing are inadequate to support 
program content; no plan exists to 
bring program content and budget 
into alignment; the 5-year budget plan 
is inadequate to support program 
expectations; the program funding 
wedge is inadequate for the formulation 
of projects beyond the 5-year horizon; 
the program and project UFE or the 
phasing of the UFE does not support 
the program and project JCLs. 1 

The program analysis schedule or 
PIMS does not consist of schedule data 
for all project effort included in the 
program scope and WBS, and with all 
effort that is under the responsibility 
of the program organization to 
perform. Schedule is not delivered 
electronically. Schedule data for all 
effort that falls under the responsibility 
of the program is not reflected in 
lower discrete detailed schedules, and 
vertical and horizontal integration is 
missing. There is missing schedule logic, 
and a program/project critical path(s) 
does not exist. Program costs are not 
mapped to the PIMS. The analysis/ 

PIMS does not follow government 
best practices and is red on schedule 
health check. The program does not 
have or is not following a Program 
Schedule Management Handbook. The 
SRB is not able to use the PIMS for risk 
identification and schedule and risk 
analysis of program risks. 

Adequacy and 
availability 
of resources 
other than 
budget 

All key implementation facilities have 
been identified and are available to 
support near-term (5-year) missions; 
staffing resource needs have been 
determined and are available; needed 
external resources are available. 

Not all key resources and facilities may 
be identified to support near-term 
(5-year) missions; known resources may 
not be available when needed; external 
resource needs are notional. 

Needed resources and/or facilities 
are not identified; availability of 
either internal or external resources is 
unknown. 
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5.1 Assessment Criteria 


Table 5-1 Example of Program Assessment Guidance (continued) 


Criteria 

Program Assessment Metrics 

Successful 


Adequacy 
of risk 

management 
approach 
and risk 
identification/ 
mitigation per 
NPR 8000.4 

The NASA continuous risk 
management paradigm is practiced. 

A knowledgeable program risk 
manager has been assigned. A 
program risk management plan 
exists and is followed; a risk database 
is being utilized to monitor, track, 
and communicate risks. Risks have 
been identified within the schedule 
with mitigation plans and are under 
configuration control. Reserves are 
adequate to manage risks. A full list of 
program or project risks — including 
title, description, mitigation plan, 
likelihood, and consequence — is 
delivered to support SRB schedule 
risk analysis, cost risk analysis, range 
estimate, and/or JCL. Uncertainty is 
mapped to cost and schedule. 

The NASA continuous risk management 
paradigm is practiced. A program 
risk manager has been assigned. 

A program risk management plan 
exists, but risk identification and/or 
mitigation is incomplete; reserves may 
not be adequate to manage risks. Risk 
management plan implementation 
is incomplete or ineffective. A list of 
program or project risks — including 
title, description, mitigation plan, 
likelihood, and consequence — is 
delivered to support SRB preliminary 
schedule risk analysis, cost risk analysis, 
range estimate and/or JCL. Uncertainty 
is mapped to cost and schedule. 

A risk management plan does not exist; 
categorization of current projects is 
inconsistent; near-term projects have 
not been categorized, projects do not 
meet classification requirements or 
are not executing risk management 
processes; no longer-term program risk 
strategy exists. 


Acronyms: JCL = Joint Confidence Level, PIMS = Program Integrated Master Schedule, UFE = Unallocated Future Expenses, 
WBS = Work Breakdown Structure. 

1 A JCL is only required at Key Decision Point (KDP) I for tightly coupled or single-project programs or by special request by the 
Convening Authorities. 


5.1. 3.1 Technical Assessments for All Projects, Single-Project 
Programs, and Tightly Coupled Programs 

The SRB conducts an independent technical assessment of the program or 
project at each LCR beginning in Formulation, continuing during Imple- 
mentation, and concluding during the Operations phase. Beginning with the 
program or project requirements, this assessment subsequently focuses on 
technical readiness, fabrication, integration, verification/validation testing, 
launch, operations, mission products, and life-cycle logistics support. 

Throughout this process, technical risk, failure tolerance, and margin 
adequacy are continually reviewed. Guidance for these assessments is found 
in the unique entrance and success criteria for each LCR in NPR 7123. IB 
Appendix G. There may be NASA Center-specific engineering processes 
and documentation that need to be included in the assessment criteria. 

Each assessment effort begins with a thorough review of the appropriate 
program or project documentation, followed by selective attendance (as 
observers) at internal project reviews. Each SRB member typically performs 
off-line analyses checks and participates in the formal LCRs. Additional meet- 
ings with project personnel may be necessary to ensure full understanding 
of complex issues and solutions. The planning and execution of these addi- 
tional meetings are defined via a coordinated effort between the SRB chair, the 
Review Manager, and the program or project manager. Each assessment should 
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5.1 Assessment Criteria 


Table 5-2 Example of Project Assessment Guidance 


Criteria 

Project Assessment Metrics 

Successful 


Alignment 
with and 
contributing 
to Agency 
needs, 
goals, and 
objectives 

Project objectives are well-aligned 
with strategic goals; project aligns with 
Level 2 requirements; objective-driven 
requirements are clearly flowed down 
through the WBS and drive the baseline 
mission design; project is in compliance 
with required NPDs and NPRs. 

Traceability of project objectives to 
strategic goals is unclear; project 
is working to align with Level 2 
requirements; requirements flow-down 
is incomplete; design capabilities are 
not yet consistent with requirements; 
project is satisfactorily working to meet 
compliance with required NPDs and 
NPRs. 

Concept capabilities are driving project 
objectives; project does not align with 
Level 2 requirements; objectives do not 
align with strategic goals; requirements 
flow-down is haphazard, without 
traceability, and/or not driving the 
design; project does not appear to meet 
compliance with NPDs and NPRs. 

Adequacy of 
management 
approach 

An effective organizational structure 
exists; management processes exist to 
effectively direct/control the project; 
essential interfaces are defined, and 
agreements are in place. 

Organizational structure is lacking 
in some areas; control processes are 
questionable or have latency issues; 
interfaces are incomplete. 

Organizational structure is 
unacceptable; necessary interfaces do 
not exist; control processes are notional 
and not in place. 

Adequacy 
of technical 
approach 

There is an acceptable baseline design; 
the design is requirements driven; 
the capabilities of the design ensure 
adequate technical margins against the 
requirements. 

The design has not yet stabilized; 
design trades remain open beyond 
expected milestones; some baseline 
design margins are inadequate against 
requirements; technical readiness is a 
concern. 

There is an inadequate baseline 
design; technical margins are clearly 
inadequate at this point in the project 
life cycle; technical maturity is unlikely 
within planned schedules. 

Adequacy 
of the 
integrated 
cost and 
schedule 
estimate 
and funding 
strategy in 
accordance 
with 

NPD 1000.5 

An adequate BOE exists for the baseline 
LCC; annual phasing fully supports 
the scheduled work content. The 
commitment baseline incorporates 
the UFE required to support the JCL; 1 
the project's management baseline 
includes an appropriate allocation of 
the UFE. 

A government project IMS reflecting 
scheduling practices, which captures 
the project's scope of work from 
the WBS in a logic network, with JCL 
required costs or detailed resources 
loading, with durations supported 
by historical projects' data and BOE, 
that is integrated horizontally and 
vertically with a valid critical path(s) 
and reasonable schedule slack 
appropriate to life-cycle phase is 
delivered electronically to support a 
SRB schedule risk analysis. Schedule 
health check is green; status is up 
to date, and approved (baseline) 
schedule is maintained. Project has 
and is implementing well-defined 
schedule management processes. 
Schedule margin, which is covered by 
an appropriate amount of UFE that is 
consistent with project schedule risk 
analysis/range estimate/JCL results. 

The BOE is incomplete or at issue for the 
baseline LCC; annual phasing partially 
supports the scheduled work content 
or is inadequate in some years. The 
commitment baseline incorporates only 
some of the UFE required to support 
the JCL; 1 the project's management 
baseline includes an inadequate 
allocation of the UFE. 

Analysis schedule or IMS partially 
captures the project's scope of work 
from the WBS in a logic network, 
with costs/resources partially loaded, 
with durations mostly supported 
by historical projects' data and BOE, 
that is integrated horizontally and 
vertically with a partially valid critical 
path(s) and reasonable schedule slack 
appropriate to life-cycle phase is 
delivered electronically to support a 
preliminary SRB schedule risk analysis. 
Schedule health check is yellow; 
government best practices need to 
be applied to the schedule; status is 
up to date, and approved (baseline) 
schedule is maintained. Project has and 
is implementing well-defined schedule 
management processes. Schedule 
margins and funded schedule margin 
are consistent with preliminary project 
schedule risk analysis/range estimate/ 
JCL results. 

The BOE is not provided or is 
substantially at issue for the baseline 
LCC; annual phasing inadequately 
supports the scheduled work content 
or is insufficient in many years. The 
commitment baseline does not 
incorporate the UFE required to support 
the JCL; 1 the project's management 
baseline does not include an allocation 
of the UFE. 

Analysis schedule or IMS does not 
reflect the project's scope of work and 
WBS; there is missing schedule logic; 
costs and resources are not loaded; 
durations are unrealistic and are not 
supported by historical data; horizontal 
and vertical integration are lacking; the 
critical path is not evident, and slack 
values are unrealistic. The schedule 
does not pass the schedule health 
check and is not viable for performing 
a schedule risk analysis, range estimate, 
or JCL calculation. The project does not 
have or is not following well-defined 
schedule management processes. 
Schedule margins and funded schedule 
margins are not justified by probabilistic 
analysis. 

Adequacy 

and 

availability 
of resources 
other than 
budget 

All resources and facilities have been 
identified and are available; resources 
are properly aligned with integrated 
cost and schedule described above; 
project is adequately staffed. 

Availability of some needed resources 
and/or facilities is questionable; staffing 
may be inadequate or lagging plan. 

Needed resources and/or facilities are 
either not identified or are not available 
within schedule and cost; staffing is 
clearly inadequate. 
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5.1 Assessment Criteria 


Table 5-2 Example of Project Assessment Guidance (continued) 


Criteria 

Project Assessment Metrics 

Successful 


Adequacy 
of risk 

management 
approach 
and risk 
identifica- 
tion/mitiga- 
tion per 
NPR 8000.4 

The NASA continuous risk management 
paradigm is practiced. A knowledgeable 
risk manager has been assigned. 

A risk management plan exists 
and is followed; a risk database is 
being utilized to monitor, track, 
and communicate risks. Risks have 
been identified within the schedule 
with mitigation plans and are under 
configuration control. Reserves are 
adequate to manage risks. A full list of 
program or project risks — including 
title, description, mitigation plan, 
likelihood, and consequence — is 
delivered to support SRB schedule 
risk analysis, cost risk analysis, range 
estimate, and/or JCL. Uncertainty is 
mapped to cost and schedule. 

The NASA continuous risk management 
paradigm is practiced. A risk manager 
has been assigned. A risk management 
plan exists, but risk identification and/ 
or mitigation is incomplete; reserves 
may not be adequate to manage risks. 
Risk management plan implementation 
is incomplete or ineffective. A list of 
program or project risks — including 
title, description, mitigation plan, 
likelihood, and consequence — is 
delivered to support SRB preliminary 
schedule risk analysis, cost risk analysis, 
range estimate, and/or JCL. Uncertainty 
is mapped to cost and schedule. 

A risk management plan does not exist 
or is incomplete; top risks have not been 
identified; not possible to determine 
adequacy of reserves to manage risks. 


Acronyms: BOE = Basis Of Estimate, LCC = Life-Cycle Cost, IMS = Integrated Master Schedule, JCL = Joint Confidence Level, 

NPD = NASA Policy Directive, NPR = NASA Procedural Requirement, UFE = Unallocated Future Expenses, WBS = Work Breakdown 
Structure. 

1 A range estimate is required at Key Decision Point (KDP) B; a JCL is only required at KDP C or by special request by the Convening 
Authorities. 


respond to issues defined in the previous LCR and identify important issues to 
be resolved before the next LCR. 

5.1. 3.2 Technical Assessments for Uncoupled or Loosely Coupled 
Programs 

For uncoupled or loosely coupled programs, the SRB technical assessments 
are characterized by specific contents defined during the initial technical 
assessment for program approval. These are then periodically reexamined 
after program acquisition in status/implementation reviews performed 
as directed by the Decision Authority. These assessments are conducted 
at a less- detailed level of engineering than project reviews since they are 
performed at a higher level. This Program Plan should typically cover a 
decade to understand the program’s strategy for pursuing Agency needs, 
goals, and objectives. Project conceptual definitions within the plan should 
be of sufficient detail to support technical and programmatic development 
plans within the program. The technical assessment also ensures that tech- 
nology readiness level maturity is consistent with the Program Plan. Each 
assessment should respond to issues defined in the previous program review 
and identify important issues to be resolved before the next status review. 
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5.1 Assessment Criteria 


The SRB uses the ICA approach to 
assess the adequacy of the budget 
and financial management practices 
to accomplish the work through 
the budget horizon. The ICA is 
comprehensive, qualitative, and 
broad in scope. The programmatic 
analysts assess the program or 
project programmatic data based on 
the planning information provided 
by the program or project. 

5.1.4 Adequacy of the Integrated Cost and Schedule 
Estimate and Funding Strategy in Accordance with 
NPD 1000.5 

Five programmatic assessment approaches are in place for ensuring that 
cost and schedule estimates and funding strategies are adequately compliant 
with NPD 1000.5, Policy for NASA Acquisition. Each approach is described 
separately in the following subsections. 

5.1.4.1 Independent Cost Analysis 

An Independent Cost Analysis (ICA) is an independent analysis of program 
or project resources. The SRB uses the ICA approach to assess the adequacy 
of the budget and financial management practices to accomplish the work 
through the budget horizon. The ICA is comprehensive, qualitative, and 
broad in scope. The programmatic analysts assess the program or project 
programmatic data based on the planning information provided by the 
program or project. A combined uncertainty and risk analysis of the 
program or project cost estimate is used to support recommendations for 
the amount of funded schedule reserve the program or project should be 
carrying in its budget plan. The ICA includes the cost estimating uncer- 
tainty inherent to the development project estimating, the program’s or proj- 
ect’s identified risks possibly adjusted by the SRB, and new risks identified 
by the SRB. The ICA results are shared with the program or project prior to 
being finalized. When available, earned value management data are used to 
generate an independent cost analysis. 

Benchmarking is used to support 
the ICA by comparing the Program 
or Project Plan with actual historical 
data or independent estimates. 

5.1.4.2 Benchmarking 

Benchmarking is used to support the ICA by comparing the Program or 
Project Plan with actual historical data or independent estimates. Bench- 
marks may be in the form of an analogy, which may be a similar program or 
project, system, subsystem, component, or activity with its actual cost and/ 
or schedule to be used for comparison with the Program or Project Plan. 
Analogies are generally applicable throughout the program or project life 
cycle. Benchmarks may also be in the form of an independent cost esti- 
mate developed by a different methodology than the program or project 
for comparison with the Program or Project Plan. Independent cost esti- 
mates are typically produced when directed by the Convening Authorities, 
at Key Decision Point (KDP) B (Mission Definition Review (MDR), System 
Definition Review (SDR)) and KDP C (Preliminary Design Review (PDR)), 
but are also generated if warranted by special circumstances to support the 
review. The intent is to use benchmarking to help substantiate the program 
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5.1 Assessment Criteria 


or project estimate and/or budget, as well as identify areas of risk that may 
affect the Program or Project Plan. 


5.1.4.3 Independent Schedule Assessment/Analysis and Schedule 
Risk Assessment per NASA/SP-201 0-3403 

Each program and project is expected to implement government best 
schedule practices as part of its responsibilities as outlined in NASA/ 
SP-2010-3403, NASA Schedule Management Handbook. The NASA Schedule 
Test and Assessment Tool (STAT) is recommended as a basis for assessing 
the schedule to ensure it meets best practices. Schedule assessment is 
the process of determining schedule validity and performance at a given 
point in time. Periodic assessment is necessary to ensure that the Inte- 
grated Master Schedule (IMS) continues to generate valid data and support 
program or project objectives throughout the program’s or project’s life 
cycle. Schedule analysis is the process of evaluating the magnitude, impact, 
and significance of actual and forecast variances to the baseline and/or 
current schedules. A Schedule Risk Assessment (SRA) is an important anal- 
ysis process that evaluates the likelihood that a project plan, reflected in the 
IMS, is achievable within the planned finish date constraints. 

A program-level schedule assessment/analysis and SRA are performed from 
a portfolio viewpoint using the Program Plan to assess the viability of the 
program planning. It includes an assessment of the program’s long-term 
alignment with sponsor goals and objectives. In tightly coupled programs, 
individual project schedules should be logically integrated into an IMS, 
allowing the SRB to assess the integrated effects across all projects and their 
impact on the program critical path. The independent schedule assessment 
will be shared with the program prior to being finalized. 

A project-level schedule assessment/analysis and SRA focus on the detailed 
implementation plan for that specific project. Various scheduling and risk 
assessment data collections are used in performing the assessment. 

The full membership of the SRB participates in schedule assessments. Using 
assessment data, the SRB can develop an understanding of the realism and 
completeness of the program or project schedule and risk areas, and identify 
where there may be inadequate phasing of available resources and resource 
availability. Additionally, the SRB will gain a better understanding of the 
risk impacts on primary, secondary, and tertiary critical paths and the rela- 
tive probability of each. 


Schedule assessment is the process 
of determining schedule validity and 
performance at a given point in time. 


Schedule analysis is the process of 
evaluating the magnitude, impact, 
and significance of actual and 
forecast variances to the baseline 
and/or current schedules. An SRA 
is an important analysis process 
that evaluates the likelihood that a 
project plan, reflected in the IMS, is 
achievable within the planned finish 
date constraints. 


The full membership of the SRB 
participates in schedule assessments. 
Using assessment data, the SRB can 
develop an understanding of the 
realism and completeness of the 
program or project schedule and risk 
areas, and identify where there may 
be inadequate phasing of available 
resources and resource availability. 
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5.1 Assessment Criteria 


The SRB is responsible for evaluating 
the submitted program or project 
cost and schedule range estimates to 
determine the quality of the product 
and acceptability of the process 
used. 

5.1.4.4 Cost and Schedule Range Estimate Assessments 

Tightly coupled and single-project programs (regardless of life-cycle cost) 
and projects that fall under the requirements of NPR 7120.5 (Section 2) shall 
provide cost and schedule ranges at KDP 0/KDP B. Each range (with confi- 
dence levels identified for the low and high values of the range) is established 
by a probabilistic analysis and based on identified resources and associated 
uncertainties by fiscal year. Separate analyses of cost and schedule, each 
with associated confidence levels, meet the requirement. A joint cost and 
schedule Joint Confidence Level (JCL) is not required, but maybe used at 
KDP 0/KDP B. 

The SRB is responsible for evaluating the submitted program or project cost 
and schedule range estimates to determine the quality of the product and 
acceptability of the process used. The SRB will incorporate any inputs iden- 
tified in the ICA and risk assessment into the program or project cost range 
and evaluate any impacts. The SRB will incorporate any inputs from the 
schedule assessment/analysis and SRA into the program or project schedule 
range estimate and evaluate any impacts. 

The SRB is responsible for analyzing 
the submitted program or project 
JCL to determine the quality of the 
product and acceptability of the 
process used. 

5.1.4.5 Cost and Schedule Joint Confidence Level Assessments 

Tightly coupled and single-project programs (regardless of life-cycle cost) 
and projects that fall under the requirements of NPD 1000.5 and Section 2 of 
NPR 7120.5 shall develop a cost- or resource-loaded schedule and perform 
a risk-informed probabilistic analysis that produces a JCL at KDP I/KDP C. 
A JCL is also required when the program or project is rebaselined. The JCL 
is the product of a probabilistic analysis of the coupled cost and schedule 
to measure the probability of completing remaining work on schedule and 
within budget levels, and on or before the planned completion of Phase D. 

The SRB is responsible for analyzing the submitted program or project JCL 
to determine the quality of the product and acceptability of the process 
used. The SRB will incorporate the inputs identified in the ICA, independent 
schedule assessment/analysis, and SRA into the program or project JCL and 
evaluate their impact. 

5.1 .5 Adequacy and Availability of Resources Other Than 
Budget 

Resources other than budget are essential elements of successful program 
functionality and project implementation and operation. These resources 
include workforce, fabrication, assembly, test facilities and equipment, test 
beds, ground support equipment, launch sites, communication networks, 
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and mission operation centers. They can be either government or privately 
held resources. 

The SRB is expected to assess the adequacy of the availability and capacity 
of these resources to meet the needs of the program or project throughout 
the life cycle. The SRB’s assessment should consider not only the adequacy of 
the proposed and acquired resources, but also alternatives that might reduce 
cost or risk or improve the performance of associated life-cycle activities. 

5.1 .6 Adequacy of Risk Management Approach and Risk 
Identification/Mitigation 

Each program or project is expected to execute a Risk Management Plan 
as part of its responsibilities. The Risk Management Plan is a plan for 
reducing risks in all mission execution domains (safety, technical, cost, and 
schedule) during all program or project phases. See NPR 8000.4, Agency 
Risk Management Procedural Requirements and NASA/SP-2011-3422, NASA 
Risk Management Handbook for further requirements and guidance on risk 
management and the PM Handbook for further guidance on addressing 
the expected maturity for each of these criteria. Program or project risk 
management entails two major processes: risk-informed decision making 
and Continuous Risk Management (CRM). The two processes are character- 
ized as follows: 

• Risk-informed decision making concerns the use of risk information 
to assist in the decision process for key decisions, such as architecture 
and design decisions, make or buy decisions, source selection in major 
procurements, and budget reallocation (allocation of reserves), which 
typically involve requirements-setting or rebaselining of requirements. It 
is divided into three major tasks: (1) identification and screening of deci- 
sion alternatives, (2) risk assessment of decision alternatives, and (3) risk- 
informed selection of the alternative to be implemented. As part of these 
tasks, risk assessment is used to evaluate the ability of each alternative to 
meet specified performance commitments within risk-tolerance limits set 
by the decision makers. 

• CRM entails the continuous management of risks to keep all perfor- 
mance risks within tolerable limits throughout all phases of Implementa- 
tion. The six main steps of CRM are to (1) identify individual risks as they 
arise, (2) analyze their effects on performance risks, (3) plan responses, 

(4) track the risk drivers, (5) control the residual risks, and (6) communi- 
cate and document the results. CRM processes are applicable at any level 
of the program or project hierarchy where performance requirements are 
defined. The CRM processes at each level are focused on achieving the 
requirements defined at that level. CRM is a dynamic activity with new 
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NPR 7120.5 and the PM Handbook 
have specific maturity expectations 
for the programs' and projects' 
products and control plans for each 
LCR. 


Risk-informed decision making 
concerns the use of risk information 
to assist in the decision process for 
key decisions. 


CRM entails the continuous 
management of risks to keep ail 
performance risks within tolerable 
limits throughout all phases of 
Implementation. 
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5.2 Maturity Matrices 


In support of the independent 
programmatic analysis, the SRB 
will engage in discussion regarding 
the assessment of project risks and 
uncertainty starting at the SRB 
kick-off meeting or no later than site 
review start date minus 60 days. SRB 
members and consultants-to-the- 
board are encouraged to use time at 
the kick-off meeting to interface with 
the program or project regarding 
questions to any risks in their areas 
of expertise. 


risk issues being added as existing risks are retired through prevention 
and mitigation responses. 

Typical performance risks of interest to the SRB would include cost over- 
runs, schedule slippage, safety mishaps, environmental impact, failure to 
achieve a needed scientific or technological objective, or failure to meet 
specified success criteria. During the life cycle, the program or project will 
maintain an integrated risk model that characterizes the performance of 
the program or project relative to requirements in these areas. The SRB is 
expected to assess the ability of the program or project risk management 
actions and plans to manage all significant threats to its success adequately. 

In support of the independent programmatic analysis, the SRB will engage 
in discussion regarding the assessment of project risks and uncertainty 
starting at the SRB kick-off meeting or no later than site review start date 
minus 60 days. SRB members and consultants-to-the-board are encouraged 
to use time at the kick-off meeting to interface with the program or project 
regarding questions to any risks in their areas of expertise. 

A final risk review meeting will be held following the program or project 
final data delivery. SRB members are expected to provide their final risk 
and uncertainty assessment to the SRB independent programmatic anal- 
yses at that time. The programmatic analysis is to be completed prior to 
the site review; however, the analysts will engage with the SRB during 
caucus sessions at the site review to ensure that the most accurate assess- 
ment of project risks and uncertainty has been captured. If any changes to 
risk scoring or uncertainty ratings need to be reflected in the analysis, these 
changes will be made at the site review. 


5.2 Maturity Matrices 

NPR 7120.5 and the PM Handbook provide maturity matrices that are a 
key component to determining if the program or project is ready to enter 
the next life-cycle phase. The matrices in NPR 7120.5 present the maturity 
expectations for the program’s or project’s control plans and milestone prod- 
ucts for each LCR. The PM Handbook provides further elaboration on the 
expected maturity state by LCR and KDP broken down by each of the Agen- 
cy’s six assessment criteria. These matrices address each type of program 
and project (uncoupled and loosely coupled programs, tightly coupled 
programs, single-project programs, and projects). SRBs use these matrices to 
guide their assessment of program or project fulfillment of the Agency’s six 
assessment criteria. 
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5.3 NPR 7123.1 Entrance and Success Criteria 

NPR 7123. IB Appendix G describes the required best practices for entrance 
and success criteria for the technical portion of the LCRs. The appendix lists 
each LCR separately and identifies the unique expectations for each review. 
The entrance criteria define the program’s or project’s expected technical 
maturity before the program or project can hold the review. The success 
criteria identify the level of technical maturity the program or project must 
have achieved before it can advance to the next development level. This 
assessment supports the Agency’s technical assessment criterion described 
in Section 5.1.3 . 

NPR 7123.1 provides guidance on the temporal importance of each of the 
entrance and success criteria for each of the program or project LCRs. As an 
example, Appendix F of this handbook provides the success criteria mapped 
onto the six assessment criteria addressed in Section 5.1. 


5.4 Requests for Action, Findings, and 
Recommendations 

5.4.1 Requests for Action 

5.4.1. 1 Program or Project Internal Reviews 

While participating in any program or project internal reviews as observers, 
the SRB chair and members may submit a Request For Action (RFA) 
through a “sponsor” — that is, a member of the internal review board. 

The RFA process used by the program or project must be a closed- loop 
process that provides tracking, disposition, and closure of the RFAs. The 
review chair of the Center’s independent internal review team and the 
program’s or project’s representative typically discuss each RFA and reach 
agreement on its merit for official acceptance as an RFA. The RFA initiator 
must be in agreement with the response before the RFA is closed. The goal is 
to have all program or project internal review RFAs closed before the SRB’s 
site review. 

5.4.1. 2 Life-Cycle Review Site Review 

The RFA process must ensure that each RFA is tracked from submission 
to closure. The program or project is responsible for RFA tracking, closure 
(with the concurrence of the initiator), and status reporting. 
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NPR 7123.1 Entrance and Success Criteria 


NPR 7123.1 has the expected 
technical maturity for both the 
entrance and success criteria for both 
programs and projects for each LCR. 


5.4 Requests for Action, Findings, and Recommendations 


Only SRB members can submit RFAs at the site review. SRB members 
submit RFAs if they believe a concern is not being addressed adequately and 
is unlikely to be resolved within the time-span of the review or more infor- 
mation is needed. The Review Manager collects all RFAs written during the 
site review and is responsible for reviewing them for clarity and scope. The 
SRB chair eliminates redundancies, rejects those that are out of scope, and 
requests rewrites if the intent or description is unclear. Before concluding 
the site review, the SRB and the program or project review the RFA list to 
determine which submittals are closed, rejected, accepted as actions, or 
accepted as advisory comments. 

It is acceptable practice for an SRB member to sponsor an RFA submitted 
by an observer or expert consultant-to-the-board at the review if he/she 
believes that the subject matter is appropriate. The SRB member is account- 
able for that RFA upon submittal. 

5.4.1. 3 Site Review RFA Closure 

The program or project provides a written response explaining how the RFA 
issue will be resolved. After reviewing the resolution, the author of the RFA 
determines whether the program or project response is satisfactory. The RFA 
author must endorse the resolution before the RFA is closed. 

If a disagreement occurs between the SRB and the program or project 
regarding closure of an RFA, attempts to resolve differences at the SRB and 
program or project level are essential. If resolution of the RFA is unobtain- 
able, information from the SRB and the program or project is elevated for 
resolution. Resolution escalates to successively higher levels of the gover- 
nance structure until resolved. 

5.4.2 Findings 

A finding is a conclusion reached based on examination or investigation. 
During the site review, SRB members document their findings according to 
the SRB chair’s guidance. A finding can be a strength or a weakness. Weak- 
nesses include issues, concerns, and observations. 

• Strength. A strength describes a feature of the program or project that in 
the judgment of the SRB is better than expected at a particular stage of 
the life cycle. It can also be an observed attribute from which the rest of 
the Agency could benefit. 

• Weakness (issue, concern, or observation). Weaknesses constitute a 
threat to the future success of the program or project. If the weakness is 
judged to be a very significant threat, it is an issue. Weaknesses that are 
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less significant threats are concerns. Observations are findings that have 
little immediate threat, but are areas to which the SRB feels the program 
or project should be sensitive. 

5.4.3 SRB Recommendation 

The SRB’s major conclusion is its determination of whether the program or 
project passed or failed the LCR assessment. The SRB provides a recommen- 
dation to the Convening Authorities to move the program or project into the 
next phase of development or hold it in the current phase. Additionally, the 
SRB offers any available recommendations for findings (issues and concerns) 
discovered during the review. If the SRB recommends that the program or 
project be passed with qualifications, it will explain the qualifications and 
rationale for advancing the program or project to the next development 
level. If the program or project does not pass, the SRB provides the reasons 
and rationale. The rationale should explain why the SRB has reservations, 
the significance of the reservations, and what corrective actions are recom- 
mended. It is not the responsibility of the SRB to determine if a delta review 
is necessary, but it may include this as part of its recommendation. The SRB 
will make a mitigation recommendation for each issue or concern that it 
brings forward to the Convening Authorities. 

The Decision Authority makes the final determination of whether a program 
or project has passed or failed the LCR and if it will be approved to progress 
to the next development phase. 


5.5 SRB Member Product 

SRB members provide the Review Manager and the SRB chair with indi- 
vidual written assessments. The Individual Member Independent Report 
(IMIR) and score card are the required format for the assessments. SRB 
members deliver a preliminary draff of the IMIR to the SRB chair prior to 
the SRB’s post-site-review discussion. The final written IMIR is due 48 hours 
after this discussion. 

The IMIR content is the member’s assessment of the program’s or proj- 
ect’s health and maturity relative to the LCR criteria. The IMIRs are used 
in reaching final SRB conclusions and archived as part of the Response, 
Recommendation, and Decision (RRD) package. 


5.5 SRB Member Product 


The SRB's major conclusion is its 
recommendation to the Convening 
Authorities. 


Each SRB member and consultant- 
to-the-board documents his/her 
assessment in a written IMIR. 
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5.6 Snapshot Report Briefing 


The SRB chair is responsible for 
presenting the snapshot report to 
the Decision Authority. 


The final SRB product is the SRB 
management briefing package with 
annotated notes, including charts 
from the independent programmatic 
analysis. 


5.6 Snapshot Report Briefing 

The snapshot report briefing takes place via a teleconference unless the 
Decision Authority requests otherwise. The Review Manager facilitates the 
discussion by briefly introducing the topic, the review milestone, and the 
key participants in the teleconference. The Review Manager introduces the 
senior manager who is chairing the meeting for any opening comments. 
The SRB chair is responsible for presenting the snapshot report. Section 4.4 
discusses the snapshot report content. The program or project is given an 
opportunity to provide responses to the SRB’s findings. 

Upon completion of the briefing, the Review Manager prepares a summary 
of any actions assigned at the briefing. The actions are captured by the 
Review Manager and sent to the participants. 


5.7 SRB Management Briefing Package 

The SRB management briefing package is usually a Microsoft PowerPoint 
presentation, with annotated notes, that reports the SRB’s assessment to the 
Convening Authorities. The SRB chair and the Review Manager develop the 
SRB management briefing package (with inputs from the SRB members) in 
compliance with established guidelines. 

This package follows the briefing sequence as described in Section 5.8 . The 
SRB chair modifies the package as he/she deems appropriate based on feed- 
back. 

The SRB management briefing (including independent programmatic 
analysis charts) package, presented to the governing Program Management 
Council (PMC), is the SRB’s final product. 


5.8 Briefings 

Briefings capture a summary of the LCR process and highlight SRB findings 
and recommendations. The briefings communicate the results of the review 
to the program or project and NASA management. The ToR identifies the 
reporting venues for each specific LCR. 

5.8.1 Initial Debriefing to Program or Project 

On the last day of the site visit, the SRB chair, with support from the SRB 
members and the Review Manager, orally briefs the program or project on 
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5.8 Briefings 


the SRB’s high-level findings. The purpose of this briefing is to inform the 
program or project of the SRB findings regarding the program’s or project’s 
issues, concerns, and strengths and to ensure that the findings are based on 
accurate data. The program or project can respond to the findings if there is 
additional data that address a finding. 

5.8.2 IPAO/OoE Quality Product Review (Dry Run) 

The SRB chair and the Review Manager prepare the SRB management 
briefing package, coordinating with Programmatic Analysis Group analysts. 
The Independent Program Assessment Office/Office of Evaluation (IPAO/ 
OoE) Quality Product Review is the initial dry run of the package. Its func- 
tion is to ensure that the management briefing package has a clear and 
concise message and complies with Agency policies. Participants receive a 
copy of the package 24-48 hours prior to the briefing. 

5.8.3 SRB Briefing to Program or Project and CMC 

After the Quality Product Review, the SRB chair and the Review Manager 
send the updated SRB management briefing package by email to the 
program or project manager, the host Center Technical Authority, and the 
Program Executive. The program or project may send comments on the 
revised briefing package to the SRB chair and the Review Manager. 

The management briefing date is coordinated by the host Center and the 
program or project. The Center Management Council (CMC) briefing 
includes the program or project responses to the SRB findings and the SRB 
recommendations on passing the program or project into the next life-cycle 
phase; and responses to all SRB recommendations, including those proposed 
to mitigate issues and concerns. The SRB briefing is presented by the SRB 
chair to the CMC or to an integrated CMC if multiple Centers are involved 
with the program or project. 

5.8.4 SRB Briefing to the DPMC 

The highest reporting level for Category 1 and 2 projects at non-KDP LCRs 
is the Directorate Program Management Council (DPMC). The timelines 
and procedures for the reporting of these projects’ LCRs should be similar in 
nature to those for Category 1 projects. (See Section 5.8.5. ) 

The SRB chair typically provides an overall pass/fail recommendation at 
the DPMC. If the DPMC is the governing PMC, this briefing should occur 
within 30 days of the review. 
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The SRB orally briefs its findings to 
the program or project and then, 
using the SRB Management Briefing 
package, briefs the management 
councils leading up to the 
appropriate governing PMC. 


5.9 KDP Decision Memorandum 


5.8.5 SRB Briefing to the APMC 

All Category 1 and program reviews are briefed to the Agency Program 
Management Council (APMC); however, the NASA Associate Administrator 
and the APMC reserve the right to request briefings on any project review. 
The briefing occurs within 30 days of the conclusion of the site review or at 
the next regularly scheduled APMC thereafter. 

The SRB management briefing package (including programmatic input) will 
be delivered in coordination with the APMC Executive prior to the APMC. 
The presentation is a coordinated effort between the program or project 
manager and the SRB chair. If required by the Decision Authority, a prebrief 
is conducted and coordinated with the APMC Executive. 


5.9 KDP Decision Memorandum 

The Decision Authority’s key decisions are summarized and recorded in the 
Decision Memorandum, signed at the conclusion of the governing PMC. 
More description of the Decision Memorandum is found in NPR 7120.5 and 
the PM Handbook. The Review Manager coordinates the OoE review of the 
Decision Memorandum prior to the governing PMC. 


5.10 Customer Surveys 

For Agency-level review, customer feedback helps to monitor and improve 
the SRB process. Surveys fall under specific customer categories: Convening 
Authorities, SRB members, and Agency customers. Surveys are requested 
after every LCR conducted by the SRB. This allows the IPAO to capture 
statistics and metrics over many reviews. The survey questions preserve the 
anonymity of the respondent. 

At the conclusion of an LCR, the Review Manager will send the survey 
information via email to the SRB members and Agency customers. 
Convening Authorities receive surveys at the discretion of IPAO manage- 
ment, but no less than once per fiscal year. 


5.11 RRD Package 

For Agency-level review, RRD packages are the official record of the LCRs 
conducted by the SRBs and kept by the IPAO. The Review Manager will 
prepare an RRD package that documents the total LCR. This is a summary 
package of existing LCR materials. The IPAO retains RRDs in an archive 
library for historical reference. 
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Appendices 



Definitions 

r\ 


Acceptable Risk. The risk that is understood and agreed to by the program 
or project, governing PMC, Mission Directorate, and other customer(s) 
such that no further specific mitigating action is required. (Some mitigating 
actions might have already occurred.) 

Acquisition. The process for obtaining the systems, research, services, 
construction, and supplies that NASA needs to fulfill its missions. Acqui- 
sition, which may include procurement (contracting for products and 
services) and begins with an idea or proposal that aligns with the NASA 
Strategic Plan and fulfills an identified need and ends with the completion of 
the program or project or the final disposition of the product or service. 

Acquisition Strategy Meeting. A forum where senior Agency management 
reviews major acquisitions in programs and projects before authorizing 
significant budget expenditures. The ASM is held at the Mission Directorate/ 
Mission Support Office level, implementing the decisions that flow out of 
the earlier Agency acquisition strategy planning. The ASM is typically held 
early in Formulation, but the timing is determined by the Mission Direc- 
torate. The ASM focuses on considerations, such as impacting the Agency 
workforce, maintaining core capabilities and make-or-buy planning, and 
supporting Center assignments and potential partners. 

Agency Baseline Commitment (ABC). Establishes and documents an inte- 
grated set of project requirements, cost, schedule, technical content, and 
an agreed-to JCT that forms the basis for NASA’s commitment with the 
external entities of Office of Management and Budget (OMB) and Congress. 
Only one official baseline exists for a NASA program or project and it is the 
ABC. 

Agency Program Management Council (APMC). The senior manage- 
ment group, chaired by the NASA AA or designee, responsible for reviewing 
Formulation performance, recommending approval, and overseeing 
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implementation of programs and Category 1 projects according to Agency 
commitments, priorities, and policies. 

Alternate Opinion. A disagreement with a recommendation or action 
resulting from a NC board that is based on a sound rationale (not on 
unyielding opposition) that an individual judges is of sufficient importance 
that it warrants a specific review and decision by higher-level management 
and the individual specifically requests that the alternate view be recorded 
and resolved by the Dissenting Opinion process. 

Approval. Authorization by a required management official to proceed with 
a proposed course of action. Approvals must be documented. 

Approval (for Implementation). The acknowledgment by the Convening 
Authority that the program or project has met stakeholder expectations and 
formulation requirements, and is ready to proceed to implementation. By 
approving a program or project, the Decision Authority commits the budget 
resources necessary to continue into implementation. Approval (for Imple- 
mentation) must be documented. 

Architecture. A term used to describe the structure and content of a NASA 
program. It is not to be confused with program roadmap, which describes 
how/when program architecture, is executed. 

Baseline (general context). An agreed-to set of requirements, cost, schedule, 
designs, documents, etc. that will have changes controlled through a formal 
approval and monitoring process. 

Baseline Design. The mission design of a project, when it is sufficiently 
mature to comply with all requirements, has an implementation and opera- 
tional schedule, and is consistent with approved/planned funding; within 
the project life cycle; the baseline design is expected at or shortly before the 
end of the formulation phase, i.e., in time for a PDR. 

Baseline Performance Review. A monthly Agency-level independent assess- 
ment to inform senior leadership of performance and progress toward 
the Agency’s mission and program or project performance. The monthly 
meeting encompasses a review of crosscutting mission support issues and all 
NASA mission areas. 

Basis Of Estimate (BOE). The documentation of the ground rules, assump- 
tions, and drivers used in developing the cost and schedule estimates, 
including applicable model inputs, rationale or justification for analogies, 
and details supporting cost and schedule estimates. The basis of estimate is 
contained in material available to the SRB and management as part of the 
LCR and KDP process. 
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Budget. A financial plan that provides a formal estimate of future revenues 
and obligations for a definite period of time for approved programs, projects, 
and activities. (See NPR 9420.1 and NPR 9470.1 for other related financial 
management terms and definitions.) 

Categorization. A means of establishing Agency expectations of PMs 
relative to oversight council and planning detail; projects are either Cate- 
gory 1, 2, or 3, with Category 1 receiving the highest level of scrutiny. (See 
Section 2.1.4 of NPR 7120. 5E for a full explanation.) 

Center Management Council (CMC). The council at a Center that performs 
oversight of programs and projects by evaluating all program or project 
work executed at that Center. 

Concern. A minor weakness or deficiency that is substantial enough to 
be worthy of note and brought to the attention of the project for mitiga- 
tion consideration, but is not a discriminator in and of itself that affects the 
ability of the project to be successful. 

Concurrence. A documented agreement by a management official that a 
proposed course of action is acceptable. 

Confidence Level. A probabilistic assessment of the level of confidence of 
achieving a specific goal. 

Configuration Management. A management discipline applied over the 
product’s life cycle to provide visibility into and to control changes to perfor- 
mance, functional, and physical characteristics. 

Conflict of Interest. A conflict of interest involves the abuse — actual, 
apparent, or potential — of the trust that NASA has in its personnel. A 
conflict of interest is a situation in which financial or other personal consid- 
erations have the potential to compromise or bias professional judgment 
and objectivity. An apparent conflict of interest is one in which a reasonable 
person would think that the individual’s judgment is likely to be compro- 
mised. A potential conflict of interest involves a situation that may develop 
into an actual conflict of interest. A conflict of interest exists whether or not 
decisions are affected by a personal interest; a conflict of interest implies 
only the potential for bias, not likelihood. 

Continuous Risk Management (CRM). A systematic and iterative process 
that efficiently identifies, analyzes, plans, tracks, controls, communicates, 
and documents risks associated with implementation of designs, plans, and 
processes. 

Convening Authority. The management official(s) responsible for convening 
a program or project review; establishing the Terms of Reference, including 
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review objectives and success criteria; appointing the SRB chair; and 
concurring in SRB membership. These officials receive the documented 
results of the review. 

Cost Analysis Data Requirement. A formal document designed to help 
managers understand the cost and cost risk of space flight projects. The Cost 
Analysis Data Requirement (CADRe) consists of a Part A “Narrative” and 
a Part B “Technical Data” in tabular form, both provided by the program 
or project or Cost Analysis Division. In addition, the project team produces 
the project life-cycle cost estimate, schedule, and risk identification, which is 
appended as Part C. 

Critical Path. A sequential path of tasks in a network schedule that repre- 
sents the longest overall duration from “time-now” through project comple- 
tion. Any slippage of the tasks in the critical path will increase the project 
duration. 

Critical Path Analysis. Analysis of the schedule critical path determines 
how long the project will take and where to best focus project management 
efforts so that the project will complete on time. It provides insight into 
which activities may need to be compressed to keep the schedule on track. 

It also provides insight into which activities have slack and can be delayed 
without impacting the project completion date. Critical path analysis 
requires constant review of the validity of the tasks, durations and logical 
relationships that are on the primary critical path, and secondary paths. 
Changes made to durations and or logical relations may shorten the critical 
path and prevent the project from slipping. 

Decision Authority (program and project context). The individual autho- 
rized by the Agency to make important decisions on programs and projects 
under this or her authority. 

Decision Memorandum. The document that summarizes the decisions 
made at KDPs or as necessary in between KDPs. The decision memorandum 
includes the Agency Baseline Commitment (if applicable), Management 
Agreement cost and schedule, UFE, and schedule margin managed above 
the project, as well as life-cycle cost and schedule estimates, as required. 

Dissenting Opinion. A Dissenting Opinion is a disagreement with a deci- 
sion or action that is based on a sound rationale (not on unyielding opposi- 
tion) that an individual judges is of sufficient importance that it warrants 
a specific review and decision by higher level management, and the indi- 
vidual specifically requests that the dissent be recorded and resolved by the 
Dissenting Opinion process. 
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Earned Value Management. A tool for measuring and assessing project 
performance through the integration of technical scope with schedule and 
cost objectives during the execution of the project. EVM provides quan- 
tification of technical progress, enabling management to gain insight into 
project status and project completion costs and schedules. Two essential 
characteristics of successful EVM are EVM system data integrity and care- 
fully targeted monthly EVM data analyses (e.g., identification of risky WBS 
elements). 

Earned Value Management System. An integrated management system and 
its related subsystems that allow for planning all work scope to completion, 
assignment of authority and responsibility at the work performance level, 
integration of the cost, schedule, and technical aspects of the work into a 
detailed baseline plan, objective measurement of progress (earned value) at 
the work performance level, accumulation and assignment of actual costs, 
analysis of variances from plans, summarization and reporting of perfor- 
mance data to higher levels of management for action, forecast of achieve- 
ment of milestones and completion of events, forecast of final costs, and 
disciplined baseline maintenance and incorporation of baseline revisions in 
a timely manner. 

Entrance Criteria. The readiness requirements imposed by NPR 7123.1 on 
program or project for all LCRs; these criteria are used as a helpful reminder 
by program or project as they prepare for each LCR. 

Evaluation. The continual self- and independent assessment of the perfor- 
mance of a program or project and incorporation of the evaluation findings 
to ensure adequacy of planning and execution according to plans. 

Final (document context). Implies the expectation of a finished product. All 
approvals required by Center policies and procedures have been obtained. 

Finding. A conclusion reached by the SRB based on examination or investi- 
gation; a finding can be a concern, issue, observation, or strength. 

Formulation. The identification of how the program or project supports 
the Agency’s strategic goals; the assessment of feasibility, technology and 
concepts; risk assessment, team building, development of operations 
concepts and acquisition strategies; establishment of high-level requirements 
and success criteria; the preparation of plans, budgets, and schedules essen- 
tial to the success of a program or project; and the establishment of control 
systems to ensure performance to those plans and alignment with current 
Agency strategies. 

Formulation Authorization Document. The document issued by the 
MDAA to authorize the formulation of a program whose goals will fulfill 
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part of the Agency’s Strategic Plan and Mission Directorate strategies and 
establish the expectations and constraints for activity in the Formula- 
tion Phase. In addition, a Formulation Authorization Document (FAD), or 
equivalent, is used to authorize the formulation of a project. 

Funding (budget authority). The authority provided by law to incur finan- 
cial obligations that will result in expenditures. There are four basic forms of 
budget authority, but only two are applicable to NASA: appropriations and 
spending authority from offsetting collections (reimbursables and working 
capital funds). Budget authority is provided or delegated to programs and 
projects through the Agency’s funds distribution process. 

Governance. The combination of processes and structures implemented by 
NASA in order to inform, direct, manage and monitor the activities of the 
organization toward the achievement of its objectives. 

Host Center. The Center with defined responsibility for a program or 
project at the Acquisition Strategy Planning (ASP) meeting and documented 
in the FAD. 

Implementation. The execution of approved plans for the development and 
operation of the program or project, and the use of control systems to ensure 
performance to approved plans and continued alignment with the Agency’s 
goals. 

Independence. Unbiased and outside the management chain of the program 
or project. The freedom from conditions that threaten objectivity or the 
appearance of objectivity. Such threats to objectivity must be managed at the 
individual reviewer and organizational levels. 

Independent Assessment(s) (includes reviews, evaluations, audits, 
analysis oversight, investigations). Assessments are independent to the 
extent the involved personnel apply their expertise impartially, without any 
conflict of interest or inappropriate interference or influence, particularly 
from the organization(s) being assessed. 

Independent Cost Analysis. An independent analysis of program or project 
resources (including budget) and financial management associated with the 
program or project content over the program’s budget horizon, conducted 
by an impartial body independent from the management of the program or 
project. ICA includes, but is not limited to, the assessment of cost estimates, 
budgets, and schedules in relation to a program or project and a program’s 
constituent Projects’ technical content, performance, and risk. ICAs may 
include ICE, assessment of resource management, distribution, and plan- 
ning, and verification of cost-estimating methodologies. (ICAs are not 
LCCEs, but are assessments of the adequacy of the budget and management 
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practices to accomplish the work scope through the budget horizon. As 
such, ICAs can be performed for program or project when a life-cycle ICE is 
not warranted.). 

Independent Cost Estimate (ICE). An independent program or project 
cost estimate prepared by an office or other entity that is not under the 
supervision, direction, or control of the program or project (or its chain of 
command) that is responsible for carrying out the development or acquisi- 
tion of the program or project. An ICE is bound by the program or project 
scope (total life cycle through all phases), schedule, technical content, risk, 
ground rules, and assumptions and is conducted with objectivity and the 
preservation of integrity of the cost estimate. ICEs are generally developed 
using parametric approaches that are tailored to reflect the design, develop- 
ment state, difficulty, and expertise of team members. 

Integrated Master Schedule. A logic network-based schedule that reflects 
the total project scope of work, traceable to the WBS, as discrete and 
measurable tasks/milestones and supporting elements that are time phased 
through the use of valid durations based on available or projected resources 
and well-defined interdependencies. 

Independent Schedule Assessment. An independent program or project 
schedule assessment prepared by an office or other entity that is not under 
the supervision, direction, or control of the program or project (or its chain 
of command) that is responsible for carrying out the development or acqui- 
sition of the program or project that includes a schedule health and quality 
check, a schedule analysis and a probabilistic schedule risk assessment. 

Issue. A deficiency or set of deficiencies taken together that are judged to 
substantially affect the ability of the project to meet their requirements 
within the planned cost and schedule. A set of deficiencies may be multiple 
concerns that taken together create a major weakness. Issues can be found 
against the project or against other organizations that affect the ability of the 
project to be successful. A major, significant weakness is an issue. 

Joint Cost and Schedule Confidence Level. (1) The probability that cost will 
be equal to or less than the targeted cost and schedule will be equal to or less 
than the targeted schedule date. (2) A process and product that helps inform 
management of the likelihood of a project’s programmatic success. (3) A 
process that combines a project’s cost, schedule, and risk into a complete 
picture. JCL is not a specific methodology (e.g., resource-loaded schedule) or 
a product from a specific tool. The JCL calculation includes consideration of 
the risk associated with all elements, regardless of whether or not they are 
funded from appropriations or managed outside of the project. JCL calcula- 
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tions include the period from KDP C through the hand over to operations, 
i.e., end of the on-orbit checkout. 

Key Decision Point (KDP). The event at which the Decision Authority 
determines the readiness of a program or project to progress to the next 
phase of the life cycle (or to the next KDP). 

Life-Cycle Cost (LCC). The total of the direct, indirect, recurring, nonre- 
curring, and other related expenses both incurred and estimated to be 
incurred in the design, development, verification, production, deployment, 
prime mission operation, maintenance, support, and disposal of a project, 
including closeout, but not extended operations. The LCC of a project or 
system can also be defined as the total cost of ownership over the project 
or system’s planned life cycle from Formulation (excluding Pre-Phase A) 
through Implementation (excluding extended operations). The LCC includes 
the cost of the launch vehicle. 

Life- Cycle Phase. The life cycle of NASA program or project is divided 
into phases, each of which defines the activities/achievements to be accom- 
plished before proceeding to the next phase; at the highest level, there are 
two phases for both programs and projects: the formulation phase, followed 
by the implementation phase. For programs the formulation phase entails 
pre-program acquisition, while the implementation phase involves program 
acquisition and operations; for projects the formulation phase entails 
pre-systems acquisition (Phases A and B), and the implementation phase 
involves system acquisition (Phases C and D), operations (Phase E), and 
decommissioning (Phase F). 

Life- Cycle Review. A review of a program or project designed to provide a 
periodic assessment of the technical and programmatic status and health 
of a program or project at a key point in the life cycle, e.g., PDR, Critical 
Design Review (CDR). Certain LCRs provide the basis for the Decision 
Authority to approve or disapprove the transition of a program or project at 
a KDP to the next life-cycle phase. 

Management Agreement. Within the Decision Memorandum, the param- 
eters and authorities over which the program or project manager has 
management control constitute the program or project Management Agree- 
ment. A program or project manager has the authority to manage within the 
Management Agreement and is accountable for compliance with the terms 
of the agreement. 

Margin. The allowances carried in budget, projected schedules, and tech- 
nical performance parameters (e.g., weight, power, or memory) to account 
for uncertainties and risks. Margins are allocated in the formulation 
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process, based on assessments of risks, and are typically consumed as the 
program or project proceeds through the life cycle. 

Metric. A measurement taken over a period of time that communicates 
vital information about the status or performance of a system, process, or 
activity. 

Mission. A major activity required to accomplish an Agency goal or to effec- 
tively pursue a scientific, technological, or engineering opportunity directly 
related to an Agency goal. Mission needs are independent of any particular 
system or technological solution. 

Mission Directorate Program Management Council (MDPMC). The 
forum that evaluates all programs and projects executed within that Mission 
Directorate and provides input to the MDAA. For programs and Category 1 
projects, the MDAA carries forward the MDPMC findings and recommen- 
dations to the APMC. 

Observation. A finding that is not substantial enough to be considered as a 
concern, but has the potential to become a concern. 

P/p. Program/project. 

Preliminary (document context). Implies that the product has received 
initial review in accordance with Center best practices. The content is 
considered correct, though some TBDs may remain. All approvals required 
by Center policies and procedures have been obtained. Major changes are 
expected. 

Program. A strategic investment by a Mission Directorate or MSO that 
has a defined architecture and/or technical approach, requirements, goals, 
objectives, funding level, and a management structure that initiates and 
directs one or more projects. A program defines a strategic direction that the 
Agency has identified as critical. 

Program Commitment Agreement. The contract between the AA and the 
responsible MDAA that authorizes transition from Formulation to Imple- 
mentation of a program. 

Program/Project Management Requirements. Requirements that focus on 
how NASA and Centers perform program and project management activi- 
ties. 

Program Plan. The document that establishes the program’s baseline for 
implementation, signed by the MDAA, Center Director(s), and program 
manager. 
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Program (Project) Team. All participants in program or project Formula- 
tion and Implementation. This includes all direct reports and others that 
support meeting program or project responsibilities. 

Programmatic Authority. Programmatic Authority includes the Mission 
Directorates and their respective program or project managers. Individ- 
uals in these organizations are the official voices for their respective areas. 
Programmatic Authority sets, oversees, and ensures conformance to appli- 
cable programmatic requirements. 

Programmatic Requirements. Requirements set by the Mission Directorate, 
program or project, and Principal Investigator, if applicable. These include 
strategic scientific and exploration requirements, system performance 
requirements, and schedule, cost, and similar non-technical constraints. 

Project. A specific investment identified in a Program Plan having defined 
requirements, a life-cycle cost, a beginning, and an end. A project also has a 
management structure and may have interfaces to other projects, agencies, 
and international partners. A project yields new or revised products that 
directly address NASA’s strategic goals. 

Project Plan. The document that establishes the project’s baseline for imple- 
mentation, signed by the responsible program manager, Center Director, 
project manager, and the MDAA, if required. 

Rebaselining. The process that results in a change to a project’s ABC. 

Request For Action (RFA). A formal written request from the SRB that asks 
for additional information from, or action by, the program or project team. 

Residual Risk. The remaining risk that exists after all mitigation actions 
have been implemented or exhausted in accordance with the risk manage- 
ment process. (See NPD 8700.1) 

Review Manager. The Review Manager has the responsibility to ensure the 
objectivity, quality, integrity, and consistency of each assigned indepen- 
dent review and will: define the scope of the review (with the Convening 
Authorities); facilitate the identification and approval of the chair and team 
members; participate on the SRB as an authority in the programmatic 
aspects (compliance to NPR 7120.5 and generally accepted rules of good 
project management, cost, schedule, and risk), and in specific technical 
areas, if appropriate; facilitate the review process; ensure that the scope of 
the review is fully exercised; and be accountable for ensuring that the results 
of the review have been properly vetted, documented and reported. 

Risk. In the context of mission execution, risk is operationally defined as 
a set of triplets: (1) The scenario(s) leading to degraded performance with 
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respect to one or more performance measures (e.g., scenarios leading to 
injury, fatality, destruction of key assets; scenarios leading to exceedance 
of mass limits; scenarios leading to cost overruns; scenarios leading to 
schedule slippage); (2) the likelihood(s) (qualitative or quantitative) of those 
scenarios; and (3) the consequence(s) (qualitative or quantitative severity of 
the performance degradation) that would result if those scenarios were to 
occur. Uncertainties are included in the evaluation of likelihoods and conse- 
quences. (See NPR 8000.4, Agency Risk Management Procedural Require- 
ments.) 

Risk Assessment. An evaluation of a risk item that determines: (1) what 
can go wrong, (2) how likely is it to occur, (3) what the consequences are, 

(4) what the uncertainties are that are associated with the likelihood and 
consequences, and (5) what the mitigation plans are. 

Risk-Informed Decision Making. A risk-informed, decision-making 
process uses a diverse set of performance measures (some of which are 
model-based risk metrics) along with other considerations within a delibera- 
tive process to inform decision making. 

Risk Management. Risk management includes risk-informed decision 
making (RIDM) and CRM in an integrated framework. RIDM informs 
systems engineering decisions through better use of risk and uncertainty 
information in selecting alternatives and establishing baseline requirements. 
CRM manages risks over the course of the development and the Imple- 
mentation Phase of the life cycle to ensure that safety, technical, cost, and 
schedule requirements are met. This is done to foster proactive risk manage- 
ment, to better inform decision making through better use of risk informa- 
tion, and then to more effectively manage Implementation risks by focusing 
the CRM process on the baseline performance requirements emerging from 
the RIDM process. (See NPR 8000.4, Agency Risk Management Procedural 
Requirements.) These processes are applied at a level of rigor commensurate 
with the complexity, cost, and criticality of the program. 

Signature. A distinctive mark, characteristic, or thing that indicates iden- 
tity; one’s name as written by oneself. 

Stakeholder. An individual or organization outside a specific program 
or project having an interest (or stake) in the outcome or deliverable of a 
program or project. 

Standards. NASA Standards are formal documents that establish a norm, 
requirement, or basis for comparison, a reference point to against which 
measure or evaluate. A technical standard, for example, establishes uniform 
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engineering or technical criteria, methods, processes, and practices. (Refer 
to NPR 7120.10, Technical Standards for NASA Programs and Projects.) 

Standing Review Board (SRB). The board responsible for conducting 
independent reviews (life cycle and special) of a program or project and 
providing objective, expert judgments to the Convening Authorities. The 
reviews are conducted in accordance with an approved ToR (See ToR 
Template, Appendix H ) and per the entrance and success criteria in 7123.1, 
the maturity matrices in the PM Handbook, and the life-cycle requirements 
in NPR 7120.5. 

Strength. A finding that describes a feature of the program or project that 
in the judgment of the SRB is better than expected at a particular stage of 
the life cycle. It can also be an observed attribute from which the rest of the 
Agency could benefit. 

Success Criteria. That portion of the top-level requirements that defines 
what must be achieved to satisfy NASA Strategic Plan objectives addressed 
by the program or project. 

System. The combination of elements that function together to produce the 
capability required to meet a need. The elements include all hardware, soft- 
ware, equipment, facilities, personnel, processes, and procedures needed for 
this purpose. 

Systems Engineering. A disciplined approach for the definition, imple- 
mentation, integration, and operation of a system (product or service). 

The emphasis is on achieving stakeholder functional, physical, and opera- 
tional performance requirements in the intended use environments over 
its planned life within cost and schedule constraints. Systems engineering 
includes the engineering processes and technical management processes 
that consider the interface relationships across all elements of the system, 
other systems, or as a part of a larger system. 

Technical Authority. Part of NASA’s system of checks and balances that 
provides independent oversight of programs and projects in support of 
safety and mission success through the selection of individuals at dele- 
gated levels of authority. These individuals are the Technical Authorities. 
Technical authority delegations are formal and traceable to the Adminis- 
trator. Individuals with Technical Authority are funded independently of a 
program or project. 

Technical Authority Requirements. Requirements invoked by OCE, 
OSMA, and OCHMO documents (e.g., NPRs or technical standards cited as 
program or project requirements) or contained in Center institutional docu- 
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ments. These requirements are the responsibility of the office or organiza- 
tion that established the requirement unless delegated elsewhere. 

Terms of Reference. A document specifying the nature, scope, schedule, 
and ground rules for an independent review or independent assessment. 

(See ToR Template, Appendix H .l 

Unallocated Future Expenses. The portion of estimated cost required to 
meet specified confidence level that cannot yet be allocated to the specific 
project WBS sub-elements because the estimate includes probabilistic risks 
and specific needs that are not known until these risks are realized. 

Uncertainty. An imperfect state of knowledge or a physical variability 
resulting from a variety of factors, including, but not limited to, lack of 
knowledge, applicability of information, physical variation, randomness 
or stochastic behavior, indeterminacy, judgment, and approximation. Also 
defined as a situation in which the outcome is subject to an uncontrollable 
event stemming from an UNKNOWN probability distribution. Schedule 
uncertainty is due to inaccurate estimates from overestimating or under- 
estimating durations (often referred to as uncertainty), changing or unad- 
dressed scope, task definition changes, and late deliveries. 

Validation. Proof that the product accomplishes the intended purpose 
based on stakeholder expectations. May be determined by a combination of 
test, analysis, demonstration, and inspection. (Answers the question, “Am I 
building the right product?”) 

Verification. Proof of compliance with design solution specifications and 
descriptive documents. May be determined by a combination of test, anal- 
ysis, demonstration, and inspection. (Answers the question, “Did I build the 
product right?”) 

Waiver. A documented authorization releasing a program or project from 
meeting a requirement after the requirement is put under configuration 
control at the level the requirement will be implemented. 

Work Breakdown Structure (WBS). A product-oriented hierarchical divi- 
sion of the hardware, software, services, and data required to produce the 
program’s or project’s end product(s), structured according to the way the 
work will be performed and reflective of the way in which program’s or proj- 
ect’s costs, schedule, technical, and risk data are to be accumulated, summa- 
rized, and reported. 
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Acronyms 


AA 

Associate Administrator 

ABC 

Agency Baseline Commitment 

APL 

Advanced Physics Laboratory 

APMC 

Agency Program Management Council 

BOE 

Basis of Estimate 

CA 

Convening Authority 

CADRe 

Cost Analysis Data Requirement 

CDR 

Critical Design Review 

CERR 

Critical Events Readiness Review 

CFR 

Code of Federal Regulations 

CMC 

Center Management Council 

CO 

Contracting Officer 

COI 

Conflict of Interest 

CRM 

Continuous Risk Management 

CS 

Civil Service Consensus Board 

CS2 

Civil Service Consensus Board with expert support 

DA 

Decision Authority 

DPMC 

Division Program Management Council 

EAG 

Evaluation and Assessment Group 

EVM 

Earned Value Management 

FAD 

Formulation Authorization Document 

FAR 

Federal Acquisition Regulation 

FRR 

Flight Readiness Review 

ICA 

Independent Cost Analysis 

ICE 

Independent Cost Estimate 

IMIR 

Individual Member Independent Report 

IMS 

Integrated Master Schedule 

IPA 

Independent Programmatic Analysis 

IPAO 

Independent Program Assessment Office 

JCL 

Joint Confidence Level 

JPL 

Jet Propulsion Laboratory 
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KDP 

Key Decision Point 

LaRC 

Langley Research Center 

LCC 

Life- Cycle Cost 

LCCE 

Life-Cycle Cost Estimate 

LCR 

Life- Cycle Review 

MD 

Mission Directorate 

MDAA 

Mission Directorate Associate Administrator 

MDR 

Mission Definition Review 

MEL 

Master Equipment List 

MRB 

Mission Readiness Briefing 

MRR 

Mission Readiness Review 

MSO 

Mission Support Office 

NASA 

National Aeronautics and Space Administration 

NC 

Non-Consensus Board 

NDA 

Non-Disclosure Agreement 

NESC 

NASA Engineering and Safety Center 

NPD 

NASA Policy Directive 

NPR 

NASA Procedural Requirement 

NSC 

NASA Safety Center 

NSCKN 

NASA Safety Center Knowledge Now 

OCC 

Office of Chief Counsel 

OCE 

Office of the Chief Engineer 

OCHMO 

Office of the Chief Health and Medical Officer 

OCI 

Organizational Conflict of Interest 

OGC 

Office of the General Counsel 

OGE 

Office of Government Ethics 

OMB 

Office of Management and Budget 

OoE 

Office of Evaluation 

ORR 

Operational Readiness Review 

OSMA 

Office of Safety & Mission Assurance 

PAG 

Programmatic Analysis Group 

PAR 

Program Approval Review 

PCI 

Personal Conflict of Interest 

PCOI 

Positional Conflict of Interest 

PDR 

Preliminary Design Review 

PE 

Program Executive 

PEL 

Power Estimate List 

PIMS 

Program Integrated Master Schedule 

PIR 

Program Implementation Review 

PLAR 

Post-Launch Assessment Review 

PM 

Program or Project Manager 

PMC 

Program Management Council 

POC 

Point of Contact 


APPENDIX B. ACRONYMS 


PPBE 

Planning, Programming, Budgeting, and Execution 

PP&C 

Program/project Planning and Control 

PRM 

Principal Review Manager 

RFA 

Request For Action 

RIDM 

Risk Informed Decision Making 

RMP 

Risk Management Plan 

RRD 

Response, Recommendation, and Decision 

S&MA 

Safety & Mission Assurance 

SBU 

Sensitive But Unclassified 

SDR 

System Definition Review 

SF 

Standard Form 

SID 

Strategic Investments Division 

SIR 

System Integration Review 

SMSR 

Safety and Mission Success Review 

SRA 

Schedule Risk Analysis 

SRB 

Standing Review Board 

SRD 

System Requirements Document 

SRR 

System Requirements Review 

STAT 

Schedule Test and Assessment Tool 

TBD 

To Be Determined 

ToR 

Terms of Reference 

TRF 

Technology Readiness Fevel 

UFE 

Unallocated Future Expenses 

WBS 

Work Breakdown Structure 


NASA STANDING REVIEW BOARD HANDBOOK 




NASA Policy on SRB 


This appendix presents the NASA Policy on Standing Review Board Composi- 
tion, Balance, and Conflicts of Interest dated December 2008. 

This Policy has been implemented since December 2008, and was issued with 
the Standing Review Board Handbook, dated November 2009. 
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The National Aeronautics and Space Administration 


POLICY ON 

STANDING REVIEW BOARD (SRB) 
COMPOSITION, BALANCE, AND CONFLICTS OF INTEREST 

December 2008 1 


Introduction 

The National Aeronautics and Space Administration (NASA) accords special importance to the policies and 
procedures established to assure the integrity of Standing Review Board (SRB) reports. The work of the SRBs 
are largely done by persons drawn from every part of the nation and from every sector of society — academia, 
industry, government, and nonprofit. The technical skills and perspectives of these individuals are essential 
to the ability of NASA to consistently produce accurate and objective assessments of NASA programs and 
projects. 

Extensive efforts are made by NASA to assure the soundness of reports by selecting highly qualified SRB 
members. Yet, if a report is to be not only sound, but also effective, the report also must be, and must be 
perceived to be, the result of a process that is generally free of bias and fairly balanced in terms of the 
knowledge, experience, and perspectives utilized to produce it. 

Questions of SRB Composition and Balance 

All individuals selected to serve on SRBs must be highly qualified in terms of knowledge, training, and 
experience — often highly specialized and particularized — to address the tasks assigned to the SRB properly. 
NASA identifies such individuals by drawing upon a network of national resources. Suggestions of potential 
SRB members come from the SRB Convening Authorities (CAs) and their staffs, from groups that have an 
interest in the underlying subject matter of a particular study and from other professionals with knowledge 
and expertise in relevant disciplines who have an interest in the programs and projects to be addressed. 

Individual qualifications are not the only determinant in this process. Having an SRB of highly qualified and 
capable individuals is necessary, but is not the only element necessary for successful reviews. When 
considering SRB membership, a well-rounded, diverse set of backgrounds can provide the most versatile 
perspective of opinions. Members should be selected both from within the Agency and from external sources, 
including such communities as private industry, academia, and other government agencies, including the 
Department of Defense (DoD). When looking internal to the Agency, various NASA Centers and cross-mission 
opportunities, e.g., robotic versus human project expertise, can add unique insights. Therefore, the 
knowledge, experience, and perspectives of potential SRB members must be thoughtfully and carefully 
assessed and balanced in terms of the subtleties and complexities of the particular scientific, technical, and 
other issues to be addressed and the functions to be performed by the SRB. Diversity and balance of 
knowledge, design/development experience and organizational experience ensures the greatest opportunity 
to provide an independent perspective. These factors should be taken into consideration when making 
recommendations for SRB membership. 


1 This Policy has been implemented since December 2008 and it is being issued with the Standing Review Board 
Handbook, dated November 2009. 


72 


APPENDIX C. NASA POLICY ON SRB 


Questions of Conflict of Interest 


The work of SRBs cannot be compromised by issues of bias and lack of objectivity. In most cases these issues 
are caused by various forms of conflicts of interest that individual SRB members may have. For purposes of 
this policy, "conflict of interest" means any financial or other interest which conflicts with the individual’s 
service on an SRB because it (1) could significantly impair the individual’s objectivity or (2) could create an 
unfair competitive advantage for any person or organization. This policy involves two different types of 
conflicts. The first type of conflict, known as an organizational conflict of interest, is based upon the interests 
of the individual’s employer. The second type of conflict, known as personal conflicts of interest, is based 
upon the personal interests of the individual. No individual that has a conflict of interest that is significant 
enough, as determined by NASA, to likely impair their judgment, relative to the functions to be performed, 
can be appointed to serve (or continue to serve) on an SRB. In some cases, such as unique expertise, it may be 
in the best interest of the government to approve potential SRB members despite the presence of conflicts of 
interest. This policy describes the process that must be followed when this occurs. 

General Principles: Organizational Conflicts of Interest 

Organizational conflicts of interest (OCI) concern the interests of the contractor for whom the individual being 
considered for service on an SRB, works. Subpart 9.5 of the FAR contains guidance on OCIs which the agency 
must follow any time the agency uses a contract to obtain the services of an individual for an SRB. The regulations 
on OCI involve the two principles: preventing the existence of conflicting roles that might bias a contractor’s 
judgment where a contractor may be a position to favor its own capabilities; and preventing unfair competitive 
advantage. There are three types of organizational conflicts of interest that emerge from these principles. 

• “Unfair access to data” occurs when a contractor has access to nonpublic information as part of its 
performance and that information may provide the firm an unfair competitive advantage in a later 
competition for a government contract. The principle of unfair competition is involved in this conflict. An 
example of this conflict involves an SRB member having access to proprietary data that could give its 
employer an unfair competitive advantage in future competitions. 

• “Biased ground rules” occurs when a contractor has the opportunity to skew a competition, whether 
intentionally or not, in favor of itself. The principles of unfair competition and bias are involved in this 
conflict. This conflict includes the interest of affiliates. An example of this conflict occurs when an SRB 
has substantial influence over a statement of work for a future competition when a member of that SRB 
intends to propose on the future competition. 

• “Impaired objectivity” involves conflicting roles that might bias a contractor’s judgment. This conflict 
contains two elements - the use of subjective judgment by the contractor and whether a contractor has a 
financial interest in the outcome of its performance. This conflict includes the interest of affiliates. The 
principle of bias is involved in this conflict. An example of this conflict occurs when an SRB member 
evaluates the work of its employer or of a competitor of its employer. 

Strategies to avoid, neutralize, or mitigate conflicts can be addressed in a formal avoidance/mitigation plan 
submitted by the contractor when required by contract. In accordance with the FAR and NFS, if the 
contracting officer determines that a certain contractor presents an OCI that cannot be effectively avoided, 
neutralized or mitigated, individuals cannot serve on an SRB absent the granting of an OCI waiver by the 
Assistant Administrator for Procurement 2 . Waivers of FAR Subpart 9.5 on organizational conflicts of interest 
will be granted on a case-by-case basis when it is determined to be in the Government’s interest to do so. 

General Principles: Personal Conflicts of Interest 

A personal conflict of interest means something more than individual bias. There must be an interest, 
ordinarily financial, that could be directly affected by the work of the SRB. 


2 This section would only apply to members on an SRB who are not civil servants. 
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Personal conflicts of interest are objective - they exist or they don’t exist. They are not an assessment of one’s 
actual behavior or character, one’s ability to act objectively despite the conflicting interest, or one’s relative 
insensitivity to particular dollar amounts of specific assets because of one’s personal wealth. Assessments of 
conflicts of interest by NASA are designed to determine if certain specific, potentially compromising 
situations might create a conflict of interest. Eliminating or preventing these conflicts of interests protect the 
individual, the other members of the SRB, NASA, and the public interest. 

Personal conflicts of interest refer to current interests. They do not apply to past interests that have expired, 
no longer exist, and cannot reasonably affect current behavior. Nor does it apply to possible interests that 
may arise in the future, but do not currently exist, because such future interests are inherently speculative 
and uncertain. For example, a pending formal or informal application for a particular job is a current interest, 
but the mere possibility that one might apply for such a job in the future is not a current interest. 

Personal conflicts of interest are not only assessed against the personal financial interests of the individual, 
but also to the interests of others with whom the individual has substantial common financial interests if these 
interests are relevant to the functions to be performed. Thus, in assessing potential personal conflicts of 
interest, consideration must be given not only to the interests of the individual, but also to the interests of the 
individual’s spouse and minor children, the individual’s business partners, and others with whom the 
individual has substantial common financial interests. Consideration must also be given to the interests of 
those for whom the individual is acting in a fiduciary or similar capacity (e.g., being an officer or director of a 
corporation, whether profit or nonprofit, or serving as a trustee). 

In assessing potential conflicts of interest in connection with an individual’s service on an SRB, particular 
attention will be given to the following kinds of financial interests if they are relevant to the program or 
projects to be reviewed and evaluated: employment relationships (including private and public sector 
employment and self-employment); consulting relationships (including commercial and professional 
consulting and service arrangements, scientific and technical advisory board memberships, and serving as an 
expert witness in litigation); stocks, bonds, and other financial instruments and investments, including 
partnerships; real estate investments; patents, copyrights, and other intellectual property interests; 
commercial business ownership and investment interests; services provided in exchange for honorariums 
and travel expense reimbursements; and research funding and other forms of research support. 

The Decision Authority has the authority to approve a written determination that a contractor’s expertise 
outweighs the contractor’s conflict of interest when the local Office of the Chief Counsel determines that a 
personal conflict of interest exists. In the case of NASA employee, only the NASA Administrator may approve 
a written determination that the employee’s expertise outweighs the employee’s personal conflict of interest. 

Access to Restricted Information 

For the purposes of this policy, "Restricted Information,” means information that is not available to the public, 
such as information developed at private expense embodying trade secrets or comprising commercial or 
financial information that is privileged or confidential; information determined by NASA to be restricted, such 
as U.S. Government Sensitive But Unclassified information as defined in NASA Procedural Requirement (NPR) 
1600.1; and "contractor bid or proposal information" or "source selection information" as defined in the FAR. 
The opportunity to have access to Restricted Information during the course of SRB activities at NASA, if 
abused or misused, may confer an unfair competitive advantage on certain contractors. Thus, individuals 
selected to serve on SRBs will be asked to sign a Non-Disclosure Agreement that provides restrictions on the 
individual’s use of Restricted Information obtained during the course of SRB activities (a model Non- 
Disclosure Agreement is attached hereto). If an individual during the course of participating in a P/p activity 
obtains and uses, or intends to use, Restricted Information for the individual’s own direct and substantial 
economic benefit, such conduct constitutes a breach of the Non-Disclosure Agreement and will be grounds for 
removal from the SRB. The same rule applies if the individual discloses, or intends to disclose, such 
information to other individuals or to organizations in such a manner that a direct and substantial economic 
benefit may be conferred on such individuals or organizations. These restrictions do not apply to information 
once it has become publicly available. 


74 


APPENDIX C. NASA POLICY ON SRB 


Employees of Sponsors 3 

There are special rules for employees of sponsors 4 . To the extent not prohibited by Federal or state laws or 
regulations, such an individual may serve as a member of such an SRB where the following requirements are 
met: (1) the service of the individual on the SRB must be based upon the unique scientific, technical or 
programmatic expertise which the individual brings to the SRB; (2) the individual and the individual’s 
supervisory chain must not be located within the chain of command for programmatic level decisions for the 
P/p; (3) it must be specifically determined during the SRB appointment process that service by the individual 
will not compromise the independence or objectivity of the review. 

Implementation of this Policy 

Background Information and Confidential Conflict of Interest Disclosures 

To address questions of SRB composition, balance and conflict of interest, individuals being considered for 
selection to serve on SRBs are required to submit certain background information, and certain information 
regarding conflicts of interest, relative to the P/p to be reviewed. The responsible independent review office 
(typically the Independent Program Assessment Office for all programs and projects with a life-cycle cost 
>$250 million) will ensure that all potential members provide the necessary information and work with 
appropriate procurement, legal and Convening Authorities in determining suitability for SRB service and 
appropriate SRB diversity and balance. To facilitate collection of this information from non-federal members, 
the "Background Information and Confidential Conflict Of Interest Disclosure" form (attached) will be used by 
appropriate contracting officers and contractors to collect the information. Disclosure of relevant 
information is a continuing obligation for the duration of the SRB for which the "Background Information and 
Confidential Conflict Of Interest Disclosure" form was prepared. If during an individual’s period of service on 
the SRB it becomes apparent to the individual that there have been changes in the information disclosed, or 
that there is new information that needs to be disclosed, such information must be reported promptly to the 
Review Manager for the P/p for which the form was completed. For proposed federal SRB members, the 
Office of Government Ethics (OGE) Form 450 or Standard Form (SF) 278 (as appropriate) will be used. 

In addition to the submission of these forms, SRBs are asked to discuss the issues of SRB composition, balance 
and conflict of interest, and the relevant circumstances of their individual members, at the first kick-off 
meeting, and annually thereafter. 

Except as required by law or court order, specific conflict of interest information obtained by NASA will be held 
in confidence by NASA. Access to such information will be limited to those offices whose proper business 
requires access to such information. Such information is not otherwise released by NASA except with the 
approval of the individual to whom the information pertains, unless release is required by law. 

Determinations on Composition, Balance and Conflicts of Interest 

The specific factors to be considered by NASA in assessing questions of SRB composition and balance will 
generally depend in each case upon the particular facts and circumstances involved. The resolution of these 
matters will be based in the final analysis upon the independent judgment of the CAs in conjunction with the 
appropriate support offices. Final authority over SRB appointments rests with the Decision Authority for the 
particular program or project under review. Flowever, nothing in this section authorizes the Convening 
Authority or Decision Authority to make determinations required by, or reserved to another official by, 
statute, regulation or NASA directive; including, without limitation, 18 U.S.C. § 201, etseq. (criminal conflict of 
interest statutes), 5 CFR Part 2635 (Standards of Conduct), 48 CFR Subpart 9.5 (Federal Acquisition 


3 For purposes of this policy, the term "sponsor” means an organization that institutionally supports the program or 
project e.g., a NASA Center or Mission Directorate. 

4 This paragraph only applies to members of an SRB who are civil servants. 
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Regulation organizational conflict of interest regulation) and 48 CFR Subpart 1809.5 (NASA FAR Supplement 
organizational and consultant conflict of interest regulation). 

Once a Convening Authority provides a list of candidates for membership that reflects the desired 
composition and balance for a particular SRB, the Review Manager will initiate the independence verification 
process to identify and analyze potential organizational and personal conflicts of interest. The list of 
candidates should include more individuals than are required to serve on an SRB to allow for alternate 
members if another candidate cannot serve due to a conflict of interest or other reason. 

For any SRB, the focus of the conflict of interest inquiry is on the identification and assessment of relationships to 
the program or projects to be reviewed and evaluated, as well as on other interests that might be directly 
affected by the review and evaluation. The concern is the individual’s objectivity while participating in the 
review and evaluation process could be impaired if that individual (or others with whom the individual has 
substantial common financial interests) has current interests, which could be directly affected by the P/p 
being evaluated. When contractors/consultants-to-the-board are or are being considered as members of 
SRBs, each member and his/her company must also be considered in the context of organizational conflicts of 
interest in relation to the program or project being independently reviewed as set forth in the FAR and the 
NFS. 

Information obtained from the "Background Information and Confidential Conflict Of Interest Disclosure" forms 
(or OGE 450/SF 278 as appropriate) and from confidential SRB discussions of SRB composition, balance and 
conflict of interest at the initial SRB meeting and annually thereafter, will be used by the responsible officials 
in addressing and resolving questions of conflict of interest (both personal and organizational). No individual 
can be appointed to serve (or continue to serve) on an SRB if NASA determines a personal conflict of interest 
exists that is significant enough to raise questions about that individual’s ability to provide unbiased advice 
and recommendations. A written determination that the need for the individual’s expertise outweighs their 
conflict of interest will be made and approved by the Decision Authorities or Administrator as part of the 
nomination process in cases where an individual has a personal conflict of interest. 

The responsible independent review office will manage the determination and maintenance of the SRB 
member independence. In accomplishing this task, contractors who provide proposed non-Federal members 
to the SRB will initiate the process of completing the "Background Information and Confidential Conflict Of 
Interest Disclosure" forms and will make an initial determination as to whether any OCI exists. In these cases, 
the support contractor will work with the responsible independent review office and the appropriate 
contracting officer to determine the degree of conflict and to devise appropriate mitigation plans. An 
assessment and determination will also be made on the existence of personal conflicts of interest and 
whether they can be eliminated or special approval obtained. Additionally, any mitigation plans or OCI 
waivers that are necessary for an individual’s participation on an SRB must be completed prior to a final 
recommendation of SRB membership to the Convening Authority. 

The responsible independent review office will review and analyze all relevant information; will finalize 
recommendations for SRB member participation and will submit a letter of nomination for the proposed SRB 
members defining the rationale for each member’s nomination. Such letter will include the disposition of any 
conflict of interest waivers or mitigation plans, and no member shall be recommended without appropriate 
resolution of any conflicts. This letter will be directed to the CAs for their approval. When changes occur that 
affect previous determinations of conflicts of interest and independence, the same process will be followed 
leading to approval or removal of SRB members. 
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Disclosure and 
N DA for Contracted 
SRB Member/ 
Consultant 


This appendix contains the following forms: 

• Background Information and Confidential Conflict of Interest Disclosure 

• Non-Disclosure Agreement 
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The National Aeronautics and Space Administration 
BACKGROUND INFORMATION AND CONFIDENTIAL CONFLICT OF INTEREST DISCLOSURE 

NAME: TELEPHONE: 

ADDRESS: 


EMAIL ADDRESS: 

CURRENT EMPLOYER: 

PROGRAM/PROJECT SRB: 


There are three parts to this form, Part I Background Information, Part II Confidential Conflict of 
Interest Disclosure, and Part III, Certification. Complete all parts, sign and date this form on the last page, and 
return the form to . Retain a copy for your records. 

PART I BACKGROUND INFORMATION 


INSTRUCTIONS 


Please provide a curriculum/resume that identifies your relevant experience, organizational 
affiliations, government service, etc. to this SRB activity. In addition, please specifically respond to the three 
specific areas identified below to facilitate an overall assessment of any biases that may exist relative to this 
SRB activity. 

I. ORGANIZATIONAL AFFILIATIONS. Report your relevant current business relationships [e.g., as an 
employee, owner, officer, director, consultant) and your relevant current remunerated or volunteer non- 
business relationships (e.g., professional organizations, trade associations, public interest or civic groups). 

II. OTHER SUPPORT. Report relevant information regarding both public and private sources of current 
support [other than your present employer), including sources of funding, equipment, facilities. 

III. ADDITIONAL INFORMATION. If there are relevant aspects of your background or present circumstances 
not addressed above that might reasonably be construed by others as affecting your judgment in matters 
within the assigned task of the SRB or panel on which you have been invited to serve, and therefore might 
constitute an actual or potential conflict of interest or source of bias, please describe them briefly. This could 
include your relationships with individuals (rather than organizations) involved in the subject of the SRB 
activity. 


SPECIFIC AFFILIATIONS, SUPPORT, AND OTHER INFORMATION: 
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PART II CONFIDENTIAL CONFLICT OF INTEREST DISCLOSURE 


INSTRUCTIONS 

It is essential that the work of SRBs not be compromised by any significant conflict of interest. For this 
purpose, the term "conflict of interest" means any financial or other interest which conflicts with the 
individual’s service on an SRB because it (1) could significantly impair the individual’s objectivity or (2) could 
create an unfair competitive advantage for any person or organization. Additional information regarding 
potential biases and conflicts of interest are provided in the NASA Policy on Standing Review Board (SRB) 
Composition, Balance and Conflicts Of Interest 

1. RELATIONSHIPS TO THE PROGRAM/PROIECTfSI BEING EVALUATED . Taking into account your 
interests and the interests of other individuals with whom you share substantial common financial interests 
(e.g., spouse, close research colleagues and collaborators, business partners] and considering the below 
prime contractors, major subcontractors, and partners involved in the Program/Projectsfs): 

Program /Project Prime Contractors, Major Subcontractors, and Partners 

List those involved 

(a) Do you or such others receive current financial support (e.g., research and/or development grants or 
contracts, procurement contracts, consulting contracts, other grant support] from the program/project(s] 
being evaluated? 

(b) Do you or such others receive substantial current non-financiai support (e.g., equipment, facilities, 
industry partnerships, research assistants and other research personnel], from the program/project(s] being 
evaluated? 

(c) Do you or such others have any other current financial interest (e.g., patent rights, interests in partnerships 
and commercial ventures] obtained from or through the program/project(s] being evaluated? 

If the answer to all of the above questions under RELATIONSHIPS TO THE 
PROGRAM/PROJECT(S) being evaluated is either "no" or "not applicable," check here (NO). 

If the answer to any of the above questions under RELATIONSHIPS TO THE 

PROGRAM/PROJECT(S) being evaluated is "yes," check here (YES), and briefly describe the 

circumstances on the last page of this form. 


2. INVESTMENT INTERESTS . Taking into account stocks, bonds, and other financial instruments and 
investments, including partnerships (but excluding broadly diversified mutual funds and any investment or 
financial interest valued at less than $15,000) - 

(a) Do you or your spouse or minor children own directly or indirectly (e.g., through a trust or an individual 
account in a pension or profit-sharing plan) any stocks, bonds or other financial instruments or investments 
that could be affected, either directly or by a direct effect on the business enterprise or activities underlying 
the investments, by the program/project being evaluated? 

(b) Do you have any other financial investments or interests, such as commercial business interests (e.g., sole 
proprietorships), investment interests (e.g., stock options), or investment relationships (e.g., involving 
parents or grandchildren) that could be affected, either directly or by a direct effect on the business 
enterprise or activities underlying the investments, by the program/project being evaluated? 

If the answer to all of the above questions under INVESTMENT INTERESTS is either "no" or 
"not applicable," check here (NO). 

If the answer to any of the above questions under INVESTMENT INTERESTS is "yes," check 
here (YES), and briefly describe the circumstances on the last page of this form. 


NASA STANDING REVIEW BOARD HANDBOOK 


79 


3. PROPERTY INTERESTS . Taking into account real estate and other tangible property interests, as well as 
intellectual property interests (e.g., patents, copyrights) - 

(a) Do you or your spouse or minor children own directly or indirectly any such property interests that could 
be directly affected by the program/project being evaluated? 

(b) To the best of your knowledge, do any others with whom you have substantial common financial interests 
(e.g., employer, business partners, relatives) own directly or indirectly any such property interests that could 
be directly affected by the program/project being evaluated? 

If the answer to all of the above questions under PROPERTY INTERESTS is either "no" or "not 
applicable," check here (NO). 

If the answer to any of the above questions under PROPERTY INTERESTS is "yes," check here 
(YES), and briefly describe the circumstances on the last page of this form. 

4. OTHER INTERESTS . 

(a) Could your current employment or self-employment (or your spouse’s current employment or self- 
employment) be directly affected by the program/project being evaluated? 

(b) T o the best of your knowledge, could any financial interests of your (or your spouse’s) employer or, if self- 
employed, your (or your spouse’s) significant clients and/or business partners be directly affected by the 
program/project being evaluated? 

(c) If you are an officer, director or trustee of any corporation or other legal entity, could the financial 
interests of that corporation or legal entity be directly affected by the program/project being evaluated? 

(d) If you are a consultant (whether full-time or part-time), could there be a direct effect on any of your 
current consulting relationships by the program/project being evaluated? 

(e) Do you have a consulting relationship with a sponsor, grantee, or contractor of the program/project being 
reviewed and evaluated that is directly related to the subject matter of the program/project review and 
evaluation for which this disclosure form is being prepared (e.g., a consulting relationship to provide 
assistance to the sponsor, grantee, or contractor with respect to the program/project review and evaluation)? 

(f) Is a central purpose of the program/project review and evaluation a critical review and evaluation of your 
own work or that of your employer? 

(g) Are you an official or employee of an agency or organization, which is a sponsor of the program/project 
that is being reviewed and evaluated and/or a sponsor of this program/project review and evaluation SRB 
activity? 

(h) Do you have any existing professional obligations (e.g., as an officer of a scientific or engineering society) 
that effectively require you to publicly defend a previously established position on an issue that is relevant to 
the functions to be performed in this SRB activity? 

(i) If you have ever been a U.S. Government employee (either civilian or military), to the best of your 
knowledge are there any federal ethics restrictions that maybe applicable to your service in connection with 
this SRB activity? 

If the answer to all of the above questions under OTHER INTERESTS is either "no" or "not 
applicable," check here (NO). 

If the answer to any of the above questions under OTHER INTERESTS is "yes," check here 

(YES), and briefly describe the circumstances below. 
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EXPLANATION OF "YES" RESPONSES fattach additional pages as necessary]: 


PART III CERTIFICATION 

If, during my period of service in connection with the activity for which this form is being completed, 
there is any change in the information I reported, or any new information that I have not reported, which needs 
to be reported, I shall report it promptly by written or electronic communication to the Program Manager. 


Signature 


Date 


Reviewed by Contract Program Manager: 


I certify that I have reviewed this disclosure form and there are no OCI/PCI's associated with 
performing work on the subject task order, or alternatively, any identified OCI/PC1 issues have been mitigated. 


Signature 


Date 
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NON DISCLOSURE AGREEMENT 


As a participant on a NASA Standing Review Board [SRB), I recognize that I may have access to information 
that is not available to the public. To the extent NASA shares such nonpublic information with me during the 
course of SRB activities, 1 agree as follows: 

1. "RESTRICTED INFORMATION," as used herein, means information to which I have access as a member of a 
NASA SRB that is not available to the public, including, but not limited to, information developed at private 
expense embodying trade secrets or comprising commercial or financial information that is privileged or 
confidential; and information determined by NASA to be restricted, such as Sensitive But Unclassified (SBU) 
information as defined in NASA Procedural Requirement (NPR] 1600.1. 

2. With respect to RESTRICTED INFORMATION, I agree that I will: 

(a) Use, disclose, or reproduce RESTRICTED INFORMATION only to the extent necessary to perform my 
duties and fulfilling my responsibilities as a member of a NASA SRB; 

(b) Safeguard RESTRICTED INFORMATION from unauthorized use, disclosure, or reproduction; 

(c) Discuss or reveal RESTRICTED INFORMATION or any information concerning SRB proceedings only 
to individuals who are participating in the same SRB proceedings, and then only to the extent such 
information is required in connection with such proceedings on a need-to-know basis; 

(d) Return or dispose of RESTRICTED INFORMATION, as NASA may direct, when the RESTRICTED 
INFORMATION is no longer needed by me for SRB activities. 

3. Notwithstanding any restriction on use, disclosure, or reproduction of RESTRICTED INFORMATION 
provided in this Agreement, I will not be restricted in the use, disclosure, or reproduction of RESTRICTED 
INFORMATION that is: 

(a) Publicly available at the time of disclosure or thereafter becomes publicly available without breach of 
this Agreement; 

(b) Known to, in the possession of, or developed by me independent of carrying out my SRB 
responsibilities and independent of any disclosure of, or without reference to, RESTRICTED INFORMATION; 

(c) Received from a third party having the right to disclose such information without restriction; or 

(d) Required to be produced or released by me pursuant to a court order or other legal requirement. 

4. If I believe that any of the events or conditions that remove restrictions on the use, disclosure, or 
reproduction of the RESTRICTED INFORMATION apply, I will promptly notify NASA of such belief prior to 
acting on such belief, and, in any event, will notify NASA prior to an unrestricted use, disclosure, or 
reproduction of such information. 

5. I understand that failure to abide by these provisions may constitute grounds for termination of my 
participation in the SRB, administrative action, and/or civil or criminal prosecution. 


YOUR SIGNATURE 


DATE 
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E Acceptable SRB 
Structures for a 
Life-Cycle Review 


Option 

CS 

CS2 

NC 

Description 

Civil Service (CS) Consensus 
Board — No Expert Support 

Civil Service Consensus Board 
with Expert Support 

Non-Consensus Mixed Board 

SRB chair 

CS 

CS 

Either CS or non-CS 

SRB Review 
Manager 

CSorJPL* 

CS orJPL* 

CSorJPL 

SRB 

composition 

CS only 

CS only; experts provide 
analyses to SRB 

Either CS or non-CS 

SRB product 

SRB produces a briefing 
package with findings of fact 
and recommendations; RFAs 
(or equivalent) from individual 
members**, chair briefs report. 

SRB produces briefing 
package with findings of 
fact and recommendations; 
RFAs (or equivalent) from any 
individual**, reports from 
individual experts**; chair briefs 
SRB report. 

Review manager assists the 
chair in assembling the briefing 
package based on inputs and 
RFAs from all individuals**; chair 
briefs personal findings and 
recommendations. 

Minority 

report 

Minority reports documented in 
SRB report and in RFAs 

Minority reports documented in 
SRB report and RFAs 

No minority report*** 

SRB 

interaction 

For CS and CS2 boards, as noted: Consensus is reached by the Civil Service board members under 
the civil service consensus (CS) and the civil service with consult support (CS2) SRB configurations. 
Consultants (non-board members) supporting CS2 boards may interact with the projects or 
programs on behalf of the SRB members to gather information used to support SRB non-deliberative 
discussions. 

For all board options: All board members can participate in open discussion with the project and 
within the SRB. Everyone can openly discuss individual points of view. 

Independence 

Normal CS ethics rules apply 

Experts providing support 
are not on the SRB. Apply 
independence standards to 
experts. 

Apply independence standards 
to experts, but allow some 
impairments, if approved. 

* JPL review managers are not members and do not have a vote. 

** Reports and RFAs can contain individual recommendations. 

*** The minority report requirements do not abridge NASA's Dissenting Opinion process per NPD 1000.0. 


SRB structure is determined on the needs of the program or project and is documented in the Terms of Reference (ToR). 
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NPR 7123. 1 to 
NPR 7120.5 
Mapping Example 


Program Life Cycle — Program Implementation Review 

NPR 7123.1 B 
(PIR success criteria) 

Assessment Criteria/WP/? 7120.5E 

Alignment 
with and 
contribution 
to Agency 
strategic goals 

Adequacy of 
management 
approach 

Adequacy 

of 

technical 

approach 

Adequacy of the 
integrated cost 
and schedule 
estimates and 
funding strategy 

Adequacy and 
availability 
of resources 
other than 
budget 

Adequacy 
of the risk 
management 
approach 

Program still meets 
Agency needs and 
should continue. 

P 






The program cost 
and schedule 
estimates are 
credible and within 
program constraints. 


S 


P 

S 

S 

Risks are identified 
and accepted by 
program/project 
leadership, as 
required. 


S 

S 

S 

S 

P 

Technical trends are 
within acceptable 
bounds. 


s 

P 



s 

Adequate progress 
has been made 
relative to plans, 
including the 
technology readiness 
levels. 


s 

P 

S 



Technologies have 
been identified 
that are ready to 
be transitioned to 
another project or 
to an organization 
outside the Agency. 


p 

s 

s 




Note: P = Primary, S = Secondary. 
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G Traceability of SRB 
Requirements in 
NPR 7120.5E to the 
SRB Handbook 


NPR 
Para # 

NPR 7120.5E Requirement Statement 

Reqm't 

Owner 

Tailor 

MD 

AA 

CD 

PM 

Comply? 

SRB 

Reqm't 

SRB 

Handbook 
Rev A 

2.2.5 

The program or project and an 
independent Standing Review Board 
(SRB) shall conduct the SRR, SDR/MDR, 
PDR, CDR, SIR, ORR, and PIR LCRs in 
Figures 2-2, 2-3, 2-4, and 2-5. 

OCE 

X 



A 


Yes 

Sections 2.2. 
23,2.4 

2.2. 5.1 

The Conflict of Interest (COI) procedures 
detailed in the NASA Standing Review 
Board Handbook shall be strictly 
adhered to. 

OCE 

X 

A 

A 

A 


Yes 

Section 3.2 

2.2. 5. 2 

The portion of the LCR conducted 
by the SRB shall be convened by the 
Convening Authorities in accordance 
with Table 2-2. 

OCE 

X 

A 

A 

A 


Yes 

Chapters 2 
and 3 

2.2.53 

The program or project manager, the 
SRB chair, and the Center Director (or 
designated Engineering Technical 
Authority representative) shall mutually 
assess the program's or project's 
expected readiness for the LCR and 
report any disagreements to the 
Decision Authority for final decision. 

OCE 

X 


A 

A 


Yes 

Section 4.2 

2.3.4 

Following each LCR, the independent 
SRB and the program or project shall 
brief the applicable management 
councils on the results of the LCR to 
support the councils' assessments. 

OCE 

X 

A 

A 

A 


Yes 

Sections 5.7. 
5.8. 5.9 


Note: This table is an excerpt of the Compliance Matrix in Appendix C of NPR 7120.5E, NASA Space Flight Program and Project Management 
Requirements modified to show how its requirements map to the discussions in the present handbook. Note that NPR 7120. 5E may have implied 
requirements that are applicable to the SRB as well. 
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Terms of Reference 
Template 


Notes to Users of This Template 

1. In addition to specifying the Terms of Reference (ToR) for reviews, this template is 
also used as final approval for the list of individuals from which Standing Review 
Board (SRB) members and consultants-to-the-board are selected. (See Section 3.4 .1 

2. This template is designed with sufficient generality to be used for both programs 
and projects. 

3. This template may be adapted to fit the special circumstances of the program or project. 

4. Statements in curly brackets and italics {italics} are explanatory notes or reminders 
and are not intended to be a part of the final ToR. 

5. Statements in straight brackets [xxx] are fields to be filled in. 

6. For tightly coupled programs and their projects, separate ToRs are not required for 
each project. The projects may be listed with the program under the description/ 
governance section. The program ToR may include the projects’ life-cycle reviews. 

7. For tightly coupled programs and their projects, separate SRBs may be structured for 
the program and each of the projects so the applicable sections of the template would 
need to be expanded to accommodate this. However, separate SRBs for each project 
are not required. There can be one SRB for a tightly coupled program and its projects. 

8. For loosely coupled or uncoupled programs, the projects under the program typi- 
cally have separate ToRs. 

9. For single-project programs, there is a single ToR. 

10. A Program Implementation Review (PIR) appendix is added to the initial ToR 
when the Decision Authority requests a PIR. 

11. The program or project manager, prior to the readiness assessment, determines if 
the review will be a one-step review or a two-step review. 

12. Use common sense to adapt the template for programs or projects to satisfy 
the review intent. For example, project “category” is not generally applicable to 
programs, and statements such as these should be eliminated. 
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Approved by: 


Concurred by: 


[Name] 

Director, Office of Evaluation 
NASA Headquarters 


[Name] {Category 1 & 2 Projects 
only} 

NASA Chief Engineer 
NASA Headquarters 


[Name] {Programs only} 

NASA Chief Engineer 
NASA Headquarters 


[Name] 

Associate Administrator, [Designated] Mission 

Directorate 

NASA Headquarters 


[Name] 

Center Director 
[Center Name] 


[Name] {Programs & Category 1 Projects 
only} 

NASA Associate Administrator 
NASA Headquarters 


90 


APPENDIX H. TERMS OF REFERENCE TEMPLATE 


Terms of Reference for the Life-Cycle Reviews 
of the [Program or Project Name] 


Document Change Log 


Document 

Version 

Date 

Prepared by 

Change Summary 
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Terms of Reference for the Life-Cycle Reviews 
of the [Program or Project Name] 


1.0 INTRODUCTION 

1.1 Purpose 

1.1.1 This Terms of Reference (ToR] describes the agreed-upon terms for the NASA life- 
cycle reviews (LCR] of the [Program/project name] that are identified in Section 6.0 
Table 6-1. 

1.1.2 The [Program/project name] LCRs are conducted to meet the intent of Agency and 
Center review processes as documented in NPR 7120.5, NASA Space Flight Program 
and Project Management Requirements, NPR 7123.1, NASA Systems Engineering 
Processes and Requirements, Space Flight Program and Project Management Handbook, 
and the Standing Review Board ( SRB } Handbook. 

1.1.3 In case of a conflict between the SRB Handbook and this ToR, this ToR takes 
precedence. 

1.2 Scope 

This ToR covers all SRB reviews for the entire life cycle of the [Program/project 
name]. Appendices are used when necessary to provide, for future reviews, details 
that may not be contained in the original ToR. {In particular, these appendices 
provide the details that are not a part of NPR 7120.5 and/or NPR 7123.1.} 

1.3 Applicable Documents 

For all documents, applicable or reference, the ToR is referenced to the document 
version extant on the approval date or the latest authorized draft version. The 
following documents include procedural requirements, specifications, and other 
special publications. The documents listed in this paragraph are applicable to the 
extent specified herein. Each LCR will be conducted under the most recently 
approved version of a listed document unless otherwise stipulated. In those 
situations where the most recently approved version is not used, the pertinent 
version is specified in this list. 

1. NPR 7120.5, NASA Space Flight Program and Project Management Requirements. 

2. NPR 7123.1, NASA Systems Engineering Processes and Requirements. 

3. NPD 1000.5, Policy for NASA Acquisition. 

4. NPR 7120.8, NASA Research and Technology Program and Project Management 
Requirements. {Only retain if used} 

5. NPR 8000.4, Agency Risk Management Procedural Requirements. 

6. {List any other specific documents you used} 

1.4 Reference Documents 

The following documents include guidelines, handbooks, and center-specific 
publications. Unless otherwise specified, the most recently approved version of a 
listed document will be used for reference during the review. 
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1. SP-2013-02-026-HQ, Standing Review Board Handbook 

2. NASA Space Flight Program and Project Management Handbook 

3. [Insert list of reference documents, e.g., 

Jet Propulsion Laboratory Institutional Project Review Plan; or 

4. Goddard Space Flight Center Review Plan ; or 

5. Marshall Space Flight Center Review Plan; or 

6. Ames Research Center Review Plan; or 

7. Cost and Schedule Handbooks, etc.] 

2.0 [PROGRAM / PROJECT NAME] DESCRIPTION AND GOVERNANCE 

2.1 The [Program/project name] [Program or project] is an [assigned mission or 
Announcement of Opportunity] [Program or Project] within the [program name] {if 
this ToR is written for a program, provide Center, division, and Mission Directorate 
information}, which is managed by [program name] Program Office at [Center name] 
for the [division name] Division of the [directorate name] Mission Directorate 
([xxMD]] of NASA. 

2.2 [Program/project name] is [hosted] (for a Program} [managed] (for a project} for 
NASA by the [Center name], [Program/project] primary goal is [key objectives of the 
mission: also brief description of Program/project]. 

2.3 The [project name] has been designated a Category [1, 2 or 3] project by NASA. The 
governing Program Management Council (PMC] is the [APMC for Category 1, MDPMC 
for Category 2 & 3], The [project name] project has been designated a Class [A, B, C, or 
D] mission in accordance with NASA procedural requirements. 

2.4 Prior to the Readiness Assessment, the [Program or project] Manager determines if 
the review is a one-step review or a two-step review. 

2.5 The agenda for any LCR is mutually agreed to by the Program/project, Program 
Executive, SRB chair, Review Manager [RM], Center representative, S&MA TA, and 
Engineering Technical Authority (ETA] (and/or designated representative]. 

2.6 The review must address any special requirements specified by the Convening 
Authorities (CA]s or Decision Authority (DA] documented in Section 5.0 of this 
document. 

2.7 For a two-step review, the first step of the review addresses the technical adequacy of 
the [Program's or project's] technical approach, and establishes the technical baseline 
taking into consideration cost and schedule. The second step of the review occurs no 
later than six months after the first step of the review and addresses all criteria 
identified in NPR 7120.5 and the success criteria in NPR 7123.1. The second step 
review is referred to as the independent integrated LCR assessment. Both reviews are 
conducted by the SRB and chaired by the SRB chair. {If agreement is different, specify 
the agreement on the chairs] 
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2.8 For a one-step review, the review is an independent review conducted by the SRB and 
chaired by the SRB chair. 

2.9 (Retain only if this applies) There are cases, particularly for human space flight 
projects, where the project uses the LCR to make formal decisions to complete the 
project's technical work and align it with the budget and schedule. In these cases the 
project manager may co-chair the LCR since the project manager is using this forum 
to make project decisions, and the SRB will conduct the independent assessment 
concurrently. The SRB chair is in total control of the SRB and can interact with the 
presenters as needed to obtain all information needed to make a full assessment of 
the [Program/project name] health and status. 

3.0 LIFE-CYCLE REVIEW CONDUCT 

3.1 The LCRs for the [Program/project name] are conducted in accordance with NPR 
7120.5, NPR 7123.1, NPD 1000.5, Center practices {include Center Practices only if 
applicable}, the SRB Handbook, and special requirements in this ToR (see Section 
5.0). Any approved waivers and deviations to NPR 7120.5 are identified in Section 

9.0. 

3.2 The SRB performs its assessment against LCR objectives and Expected Maturity 
States defined in NPR 7120.5, Space Flight Program and Project Management 
Handbook, and NPR 7123.1. All approved requirement changes and additional 
requirements listed in Section 5.0 and waivers and deviations listed in Section 9.0 
are integrated into the assessment criteria. 

3.3 Special LCR requirements from the CAs or the [Program/project name] are identified 
in Section 5.0. 


4.0 SRB PARTICIPANTS APPROVAL AND SRB OPERATIONS 

4.1 The selection of SRB members and consultants-to-the-board is conducted in 
accordance with the SRB Handbook. The Conflict of Interest component is addressed 
in both NPR 7120.5 and the SRB Handbook. The SRB operations are conducted in 
accordance with NPR 7120.5 and the SRB Handbook. 

4.2 The skills matrix in attachment 2 presents a complete list of individuals approved to 
participate on any SRB associated with the [Program/project name]. The biography 
for each individual is provided in attachment 1. The SRB skills matrix identifies the 
primary and secondary skills of the individuals covering the SRB chair, Review 
Manager, SRB members, and consultants-to-the-board. The signing of this ToR is 
approval of these individuals for participation on the [Program/project name] SRB. 

4.3 In accordance with procedures for determining SRB members' and consultants-to- 
the-board's suitability for service, the following actions have been taken: civil 
servants have been vetted for personal and positional conflict of interest (COI) and no 
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conflicts were identified; contractors acting as SRB members or consultants-to-the- 
board have been vetted for both organizational conflict of interest (OCI] and personal 
conflict of interest (PCI] by their respective contracting officers/legal offices and have 
been certified as being free from conflict or have an approved waiver. Contractors 
have signed nondisclosure agreements. Based on the composition of the proposed 
SRB, the review process is conducted as a [insert board type, i.e., consensus board 
(CS], consensus board with consultant-to-the-board support board (CS2] or non- 
consensus board (NC]]. 

4.4 Standard program or project data and information required for the programmatic 
assessment are listed in section 7.0, Table 7-1 with required timelines for their 
delivery to the SRB. 

4.5 LCR assessment criteria are identified in NPR 7120.5, NPR 7123.1, and the PM 
Handbook. The SRB products are specified in the SRB Handbook. 

4.6 For Agency-level SRBs, after the readiness assessment and prior to an individual LCR, 
the SRB chair sends an email to the Director of the Independent Program Assessment 
Office (IPAO) stating his/her conclusions and approval/disapproval of the 
Program/project's readiness-to-proceed. The IPAO Director then sends an email to 
the CAs containing the following information: 

1. Attendees at the Readiness Assessment 

2. Results of the readiness assessment. 

3. LCR specific information (as required] 

4. Agenda of the upcoming LCR. 

5. LCR timeline. 

6. List of SRB members and consultants-to-the-board that will participate in the 
LCR. 

For Center-level SRBs - The SRB chair follows the process and practice defined by the 
Center for reporting his readiness assessment. 

4.7 The Operational Readiness Review (ORR] is the last LCR that is conducted by the 
SRB. All LCRs post ORR are institutionally (i.e., center] convened reviews. The SRB 
will be disbanded and charge codes/task orders for all SRB members and consultants- 
to-the-board (except the chair] will be closed. The SRB chair will be kept on contract 
through the launch of [Program/project name], 

5.0 SPECIAL/ADDITIONAL REQUIREMENTS, SUCCESS CRITERIA 

AND ASSESSMENT PRODUCTS 

{The authorizing documentation for these changes is essential. 

List the additions and the requestor in the sections below and provide the authorizing 

information in an attachment to this ToR.} 

5.1 General additions (entrance criteria, success criteria, etc.] requested by the CAs. 
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[Change description] [Change requestor's name] 

5.2 Any additions documented in the [Program/project name] [Program/project] plan. 
[Change description] [Change requestor's name] 

6.0 LIFE-CYCLE REVIEW PLANNING 

The SRB conducts independent reviews at the life-cycle milestones defined in NPR 7120.5. 
Table 6.1 lists the LCRs requiring an independent review by the SRB. 


Table 6-1. Listing of Life-Cycle Reviews {projects} 


Life-Cycle Review 

Review Date* 

Mission Definition Review (MDR]/System Requirements Review 
(SRR) KDP-B 

Feb 2022 

Preliminary Design Review (PDR) KDP-C 

Feb 2023 

Critical Design Review (CDR] 

Feb 2024 

System Integration Review (SIR] KDP-D 

Feb 2025 

Operations Readiness Review (ORR] 

Feb 2026 




*Note: Review dates are estimates and subject to change. 

{Or} 


Table 6-1. Listing of Life-Cycle Reviews {Programs} 


Life-Cycle Review 

Review Date 

Program Approval Review (PAR] KDP 0/1 

Feb 2022 

Program Implementation Review 1* (PIR] KDP-II 

* 




* Subsequent PIRs/KDPs will be conducted as required by the APMC. 


7.0 STANDARD PROGRAM/PROJECT LIFE-CYCLE REVIEW 
DELIVERABLES TO THE SRB 

7.1 The cost, schedule, technical, and risk data required to support an SRB programmatic 
assessment is required at three points: data access and then two data deliveries as 
shown in Table 7-1. Data access is for the program/project to provide existing data to 
the SRB to help inform and educate the SRB members. This enables the SRB to 
provide early feedback on the health of the schedule and cost data, which allows the 
program/project an opportunity to correct any potential problems areas before the 
site review. The first data delivery is the preliminary data required for the SRB 
assessment, including the delivery of any applicable preliminary models. The second 
data delivery is the final set of data for the SRB assessment before the site review. 

The data requested is intended to be that used by the program/project in doing their 
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planning and implementation and should not necessitate developing separate, new 
deliverables for the SRB. 

7.2 All other data are to be provided no later than 20 days prior to the site review. 


Table 7-1. Life-Cycle Review Data Deliveries {Projects or tightly-coupled Programs} 


Item 

Content 

Timeline 

Data Access* 

Existing Program/project management documentation (ref. NPR 
7120.5E, Tables 1-2 - 1-6), including working technical baseline 
description; project risk list, matrix and mitigation plans; Work 
Breakdown Structure (WBS), WBS dictionary; Master Equipment List; 
Equipment Power Consumption List; software lines of code, 
Integrated Master Schedule (IMS); cost estimate and planning budget 
by year and phase; staffing requirements and plans; and 
infrastructure requirements. 

100 calendar 
days prior to 
LCR** 

Data Delivery 1* 

Preliminary delivery of data formally required for the review, 
including Basis of Estimates (BOEs) for cost and schedule, a cost and 
schedule range estimate or functional Joint cost and schedule 
Confidence Level (JCL) model and analysis schedule (if required for 
LCR) and supporting data (as applicable), and/or any updates that 
have been made to the risk list, matrix, cost estimate, budget and 
schedule. 

60 calendar 
days prior to 
LCR** 

Data Delivery 2* 

Final range estimate or JCL model and analysis schedule (if range/JCL 
required) and/or any updates that have been made to the risk list, 
matrix, cost estimate, budget, schedule and P/p documents. 

20 calendar 
days prior to 
LCR** 


* The list of the programmatic cost and schedule data for each independent LCR is found in the SRB 
Handbook. 

** For two-step LCRs, the timeline is with respect to the second step of the independent LCR. 

{Or} 

Table 7-1. Life-Cycle Review Data Deliveries {Uncoupled and Loosely Coupled 
Programs} 


Item 

Content 

Timeline 

Data Access* 

Existing Program/project management documentation (ref. NPR 
7120. 5E, Tables 1-1), including working technical baseline 
description; program risk list, matrix and mitigation plans; WBS, WBS 
dictionary; 1 MS; cost estimate and planning budget by year and phase; 
staffing requirements and plans and infrastructure requirements. 

100 calendar 
days prior to 
LCR** 

Data Delivery 1* 

Preliminary delivery of data formally required for the review, 
including BOEs for cost and schedule and supporting data (as 
applicable); and/or any updates that have been made to the risk list, 
matrix, cost estimate, budget and schedule. 

60 calendar 
days prior to 
LCR** 

Data Delivery 2* 

Final budget and schedule and supporting data (as applicable) and/or 
any updates that have been made to the risk list, matrix, cost estimate, 
schedule and P/p documents. 

20 calendar 
days prior to 
LCR** 


* The list of the programmatic cost and schedule data for each independent LCR is found in the SRB 
Handbook. 

** For two-step LCRs, the timeline is with respect to the second step of the independent LCR. 
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Terms of Reference for the Life-Cycle Reviews 
of the [Program or Project Name] 


9.0 APPROVED WAIVERS AND DEVIATIONS {List in this section all waivers 
and deviations to assessment criteria and review process. Include the official 
documentation (copy) authorizing each waiver and deviation in Attachment 3 to 
this ToR. Changes in listings and /or attachments do not constitute a change to this 
ToR and do not require approval or signatures.} 

[Waiver or deviation description] [Requestor's Name] 
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Attachment 1. SRB Membership and Biographies 

Upon approval of the ToR, the following list of individuals shall become the source from 
which members and consultants-to-the-board are selected to support [Program/project 
name] SRB for the LCRs. Additional individuals can be added to the list through future 
revisions to the ToR or through joint approval by the CAs through IPAO-facilitated email 
exchange with the appropriate representatives from each organization included in the 
distribution. Any additional individuals who are approved are added to the established 
NASA-approved list from which review-specific SRB members and consultants-to-the- 
board are selected by the [Program/project name] SRB chair (not to exceed 12 members at 
each review whenever possible]. The selected members and consultants-to-the-board for 
any upcoming review will be published in the readiness assessment email prepared by the 
IPAO Director to the Convening Authorities of the [project name] SRB in advance of the 
review. 

The chair and Review Manager were approved by letter dated [month day, 20XX], 

The following individuals’ biographies for the [Program/project name] SRB are provided 
below. The individuals are approved with the signing of this ToR. 

{Include only what is applicable if any] 

The original SRB members and consultants-to-the-board were approved (select which is 
appropriate} [with the original ToR] [by the SRB approval letter] dated [XXX], Since that 
approval, a new (select which is appropriate} [individual] [RM] was approved by the 
change approval letter dated XXX. All initial approval letters and change approval letters 
are located in the IPAO. 


SRB Members Biographies 
Mr. Adam Public, Center Name, SRB Chair 

Mr. Public has over 35 years of experience as a developer. During his career he has served 
several positions on Voyager and as Spacecraft Systems Engineering Section Mission 
Manager (SSESMM], Mr. Public has received several NASA awards, including the NASA 
Service Medal. 

Mr. Public received his Bachelor of Science in Aerospace Engineering from State University 
and his Master of Science in Aerospace Engineering from a University in California. 
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Dr. Susan Jackson, Center Name, Instrument Systems 

Dr. Jackson has 25 years of experience in spacecraft and instrument systems. She is 
currently working in the Systems Office as a Review Assistant. She worked for the Project 
supporting the Space Environment In-Situ Suite (SEISS) instruments for technical design 
calibration testing. She also led the Group to produce the simulated data satellite. 

Dr. Jackson received her Bachelor of Science in Design Engineering from Some University, 
her Master of Science in Design Engineering from a University in North Dakota, and her 
Doctorate of Design Engineering from the University of a State. 

{Continue adding biographies until all members are included.} 
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SRB Consultants-to-the-Board Biographies 

Mr. Adam Public, ABD Company Name, Cryogenics 

Mr. Public has over 35 years of experience as a developer. During his career he has served 
several positions on Voyager and as Spacecraft Systems Engineering Section Mission 
Manager (SSESMM). Before retiring from NASA, Mr. Public received several NASA awards, 
including the NASA Service Medal. Mr. Public is currently working for ABD Company as a 
Cryogenics Specialist. He is also an adjunct professor at his hometown's Community 
College. Mr. Public is a leader in the .... 

Mr. Public received his Bachelor of Science in Aerospace Engineering from State University 
and his Master of Science in Aerospace Engineering from a University in California. 

Dr. Anna Smith, XYZ Company Name, Verification and Validation 

Dr. Smith has years of experience in verification and validation (V&V). She is currently 
working in the Office Group as a Reviewer. She worked for the Project supporting the 
Space Environment Suite (SIS] instruments for technical calibration testing. She also led 
the effort to produce the simulated data satellite. 

Dr. Smith received her Bachelor of Science in Design Engineering from The State University, 
her Master of Science in Design Engineering from a University in Florida, and her Doctorate 
of Design Engineering from the a State Private College. 

{Continue adding biographies until all consultants-to-the-board are included.} 
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{Include this Attachment only if you have a Waiver or Deviation per Section 9.0; 
Attach a copy of the official documentation supporting each waiver and deviation} 

Attachment 3. Waiver and Deviation Documentation 
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Reference 

Documents 


NPD 1000. OA, NASA Governance and Strategic Management Handbook 
NPD 1000. 5B, Policy for NASA Acquisition 
NPR 1900. 3B, Ethics Program Management 

NPR 7120. 5E, NASA Space Flight Program and Project Management 
Requirements 

NPR 7123. IB, NASA Systems Engineering Processes and Requirements 

NPR 8000.4, Agency Risk Management Procedural Requirements 

NASA/SP-2014-3705, NASA Space Flight Program and Project Management 
Handbook 

NASA/SP-2010-3403, NASA Schedule Management Handbook 
NASA/SP-201 1-3422, NASA Risk Management Handbook 
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